[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:57:25 dist-test-slave-gp6n krb5kdc[20814](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:57:25 dist-test-slave-gp6n krb5kdc[20814](info): set up 2 sockets
Aug 10 00:57:25 dist-test-slave-gp6n krb5kdc[20814](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 00:57:26.974080 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.003s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 00:57:27 dist-test-slave-gp6n krb5kdc[20814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:57:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:57:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 00:57:27.179893 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:41269
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:41269
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:27.537612 20829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:27.537972 20829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:27.538156 20829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:27.545292 20829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:57:27.545439 20829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:27.545526 20829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:27.545611 20829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:57:27.545678 20829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:57:27.555256 20829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:41269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:41269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:27.556617 20829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:27.557981 20829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:27.570729 20834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:27.570686 20835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:27.570637 20838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:27.571137 20829 server_base.cc:1034] running on GCE node
I20260810 00:57:27.572142 20829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:27.573711 20829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:27.574959 20829 hybrid_clock.cc:648] HybridClock initialized: now 1786323447574900 us; error 54 us; skew 500 ppm
I20260810 00:57:27.575320 20829 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:27 dist-test-slave-gp6n krb5kdc[20814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323447, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:27.578965 20829 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 00:57:27.580557 20829 webserver.cc:460] Webserver started at http://127.20.81.190:32827/ using document root <none> and password file <none>
I20260810 00:57:27.581254 20829 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:27.581406 20829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:27.581715 20829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:27.584757 20829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "6e742340431440e6911986debd7d9694"
format_stamp: "Formatted at 2026-08-10 00:57:27 on dist-test-slave-gp6n"
server_key: "530aa3eef75c26a4c34a61fb5d505e44"
server_key_iv: "9c139dadc89e52c9f770c92468bddd32"
server_key_version: "encryptionkey@0"
I20260810 00:57:27.585543 20829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "6e742340431440e6911986debd7d9694"
format_stamp: "Formatted at 2026-08-10 00:57:27 on dist-test-slave-gp6n"
server_key: "530aa3eef75c26a4c34a61fb5d505e44"
server_key_iv: "9c139dadc89e52c9f770c92468bddd32"
server_key_version: "encryptionkey@0"
I20260810 00:57:27.590828 20829 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 00:57:27.594290 20847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:27.595654 20829 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:57:27.595831 20829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "6e742340431440e6911986debd7d9694"
format_stamp: "Formatted at 2026-08-10 00:57:27 on dist-test-slave-gp6n"
server_key: "530aa3eef75c26a4c34a61fb5d505e44"
server_key_iv: "9c139dadc89e52c9f770c92468bddd32"
server_key_version: "encryptionkey@0"
I20260810 00:57:27.596068 20829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:27.615585 20829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:27.620342 20829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:27.620678 20829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:27.641057 20829 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:41269
I20260810 00:57:27.641096 20904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:41269 every 8 connection(s)
I20260810 00:57:27.642572 20829 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:57:27.647102 20905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:27.650511 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 20829
I20260810 00:57:27.650880 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 00:57:27.653692 20806 external_mini_cluster.cc:1428] Setting key 792089c4dd760c8ee9604bd1777a746e
I20260810 00:57:27.663057 20905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694: Bootstrap starting.
I20260810 00:57:27.666272 20905 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:27.667486 20905 log.cc:824] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:27.670697 20905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694: No bootstrap required, opened a new log
I20260810 00:57:27.678735 20905 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e742340431440e6911986debd7d9694" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 41269 } }
I20260810 00:57:27.679255 20905 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:27.679402 20905 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e742340431440e6911986debd7d9694, State: Initialized, Role: FOLLOWER
I20260810 00:57:27.680083 20905 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e742340431440e6911986debd7d9694" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 41269 } }
I20260810 00:57:27.680313 20905 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:57:27.680482 20905 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:57:27.680675 20905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694 [term 0 FOLLOWER]: Advancing to term 1
Aug 10 00:57:27 dist-test-slave-gp6n krb5kdc[20814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:27.683831 20905 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e742340431440e6911986debd7d9694" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 41269 } }
I20260810 00:57:27.684508 20905 leader_election.cc:302] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e742340431440e6911986debd7d9694; no voters:
I20260810 00:57:27.685184 20905 leader_election.cc:288] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:57:27.685338 20911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:57:27.687711 20905 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:57:27.687662 20911 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694 [term 1 LEADER]: Becoming Leader. State: Replica: 6e742340431440e6911986debd7d9694, State: Running, Role: LEADER
I20260810 00:57:27.688534 20911 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e742340431440e6911986debd7d9694" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 41269 } }
I20260810 00:57:27.694236 20914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e742340431440e6911986debd7d9694. Latest consensus state: current_term: 1 leader_uuid: "6e742340431440e6911986debd7d9694" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e742340431440e6911986debd7d9694" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 41269 } } }
I20260810 00:57:27.694908 20914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694 [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:27.694857 20913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e742340431440e6911986debd7d9694" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e742340431440e6911986debd7d9694" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 41269 } } }
I20260810 00:57:27.695252 20913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694 [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:27.698872 20909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:27.658926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35406 (local address 127.20.81.190:41269)
0810 00:57:27.661041 (+ 2115us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:27.661058 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:27.661111 (+ 53us) server_negotiation.cc:407] Connection header received
0810 00:57:27.669761 (+ 8650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:27.669781 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:27.670124 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:27.670509 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:27.673528 (+ 3019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:27.674962 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:27.676617 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:27.677315 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:27.677551 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:27.684573 (+ 7022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:27.684581 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:27.684595 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:27.684629 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:27.690624 (+ 5995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:27.692161 (+ 1537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:27.692175 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:27.692188 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:27.692478 (+ 290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:27.693355 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:27.693361 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:27.693368 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:27.694672 (+ 1304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:27.695229 (+ 557us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:27.696632 (+ 1403us) server_negotiation.cc:299] Negotiation successful
0810 00:57:27.697198 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1868,"thread_start_us":1610,"threads_started":1}
I20260810 00:57:27.701313 20922 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:57:27.707249 20922 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:57:27.717564 20922 catalog_manager.cc:1347] Generated new cluster ID: 7ba3c8e20a3c4037a2a5d99f83f31724
I20260810 00:57:27.717702 20922 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:57:27.735136 20922 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:57:27.736696 20922 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:57:27.748989 20922 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e742340431440e6911986debd7d9694: Generated new TSK 0
I20260810 00:57:27.749872 20922 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.81.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
I20260810 00:57:27.806197 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:41269
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:28.175248 20935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:28.175563 20935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:28.175666 20935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:28.183185 20935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:28.183305 20935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:28.183422 20935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 00:57:28.193495 20935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:41269
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:28.194772 20935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:28.196102 20935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:28.210095 20940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:28.210042 20943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:28.210117 20941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:28.211196 20935 server_base.cc:1034] running on GCE node
I20260810 00:57:28.211807 20935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:28.212733 20935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:28.213969 20935 hybrid_clock.cc:648] HybridClock initialized: now 1786323448213913 us; error 38 us; skew 500 ppm
I20260810 00:57:28.214283 20935 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:28 dist-test-slave-gp6n krb5kdc[20814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323448, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:28.217705 20935 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 00:57:28.219687 20935 webserver.cc:460] Webserver started at http://127.20.81.129:43451/ using document root <none> and password file <none>
I20260810 00:57:28.220394 20935 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:28.220569 20935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:28.221001 20935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:28.225167 20935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "3fee639ae2324e659151bbad2a618e75"
format_stamp: "Formatted at 2026-08-10 00:57:28 on dist-test-slave-gp6n"
server_key: "3ef0ffe13c61e3b2ec15788d5ab68b2d"
server_key_iv: "d1ab786b40614453610886d58fa8a421"
server_key_version: "encryptionkey@0"
I20260810 00:57:28.226186 20935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "3fee639ae2324e659151bbad2a618e75"
format_stamp: "Formatted at 2026-08-10 00:57:28 on dist-test-slave-gp6n"
server_key: "3ef0ffe13c61e3b2ec15788d5ab68b2d"
server_key_iv: "d1ab786b40614453610886d58fa8a421"
server_key_version: "encryptionkey@0"
I20260810 00:57:28.231745 20935 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 00:57:28.235249 20950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:28.236529 20935 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260810 00:57:28.236704 20935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "3fee639ae2324e659151bbad2a618e75"
format_stamp: "Formatted at 2026-08-10 00:57:28 on dist-test-slave-gp6n"
server_key: "3ef0ffe13c61e3b2ec15788d5ab68b2d"
server_key_iv: "d1ab786b40614453610886d58fa8a421"
server_key_version: "encryptionkey@0"
I20260810 00:57:28.236901 20935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:28.286248 20935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:28.291066 20935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:28.291416 20935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:28.292491 20935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:57:28.293943 20935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:57:28.294060 20935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:28.294245 20935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:57:28.294333 20935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:28.333292 20935 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:34453
I20260810 00:57:28.333339 21075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:34453 every 8 connection(s)
I20260810 00:57:28.334838 20935 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 00:57:28.339068 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 20935
I20260810 00:57:28.339319 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 00:57:28.340034 20806 external_mini_cluster.cc:1428] Setting key 14dad5cb164bc998c63f52a7709ca107
Aug 10 00:57:28 dist-test-slave-gp6n krb5kdc[20814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323448, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:28.360176 21080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:28.337512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:48523 (local address 127.20.81.190:41269)
0810 00:57:28.338038 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:28.338046 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:28.339169 (+ 1123us) server_negotiation.cc:407] Connection header received
0810 00:57:28.341648 (+ 2479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:28.341654 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:28.341752 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:28.341872 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:28.343619 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:28.345273 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:28.347609 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:28.348034 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:28.348191 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:28.352576 (+ 4385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:28.352585 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:28.352594 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:28.352641 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:28.356344 (+ 3703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:28.357274 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:28.357282 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:28.357288 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:28.357443 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:28.357923 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:28.357931 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:28.357937 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:28.358333 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:28.358521 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:28.359408 (+ 887us) server_negotiation.cc:299] Negotiation successful
0810 00:57:28.359666 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":231,"threads_started":1}
I20260810 00:57:28.360755 21081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:28.338095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:41269 (local address 127.20.81.129:48523)
0810 00:57:28.338981 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:28.339025 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:28.341310 (+ 2285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:28.342174 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:28.342183 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:28.342812 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:28.343380 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:28.343397 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:28.345607 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:28.345611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:28.347431 (+ 1820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:28.347468 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:28.348336 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:28.348346 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:28.348624 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:28.349464 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:28.349502 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:28.352319 (+ 2817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:28.356505 (+ 4186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:28.356512 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:28.356525 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:28.357129 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:28.357567 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:28.357570 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:28.357573 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:28.357773 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:28.358528 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:28.358534 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:28.358892 (+ 358us) client_negotiation.cc:769] Sending connection context
0810 00:57:28.359219 (+ 327us) client_negotiation.cc:240] Negotiation successful
0810 00:57:28.359644 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":382,"threads_started":1}
I20260810 00:57:28.362365 21078 heartbeater.cc:347] Connected to a master server at 127.20.81.190:41269
I20260810 00:57:28.362722 21078 heartbeater.cc:464] Registering TS with master...
I20260810 00:57:28.363704 21078 heartbeater.cc:511] Master 127.20.81.190:41269 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.81.130@KRBTEST.COM; defaulting to no policy
I20260810 00:57:28.366111 20865 ts_manager.cc:194] Registered new tserver with Master: 3fee639ae2324e659151bbad2a618e75 (127.20.81.129:34453)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.130@KRBTEST.COM" created.
I20260810 00:57:28.368304 20865 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:48523
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
I20260810 00:57:28.393205 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:41269
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:28.726706 21085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:28.727031 21085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:28.727195 21085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:28.734519 21085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:28.734654 21085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:28.734849 21085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 00:57:28.744678 21085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:41269
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:28.745954 21085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:28.747347 21085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:28.760875 21094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:28.760969 21092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:28.760973 21091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:28.762022 21085 server_base.cc:1034] running on GCE node
I20260810 00:57:28.762673 21085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:28.763530 21085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:28.764747 21085 hybrid_clock.cc:648] HybridClock initialized: now 1786323448764707 us; error 24 us; skew 500 ppm
I20260810 00:57:28.765089 21085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:28 dist-test-slave-gp6n krb5kdc[20814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323448, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:28.773136 21085 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 00:57:28.774639 21085 webserver.cc:460] Webserver started at http://127.20.81.130:35773/ using document root <none> and password file <none>
I20260810 00:57:28.775318 21085 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:28.775419 21085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:28.775699 21085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:28.778707 21085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "43e6ae21a7314c1d8c305bebc1f0e7e7"
format_stamp: "Formatted at 2026-08-10 00:57:28 on dist-test-slave-gp6n"
server_key: "26223cf49929c47fcae6bd13669bd15c"
server_key_iv: "45f240b6e3ec2033bb72dd57d493aee8"
server_key_version: "encryptionkey@0"
I20260810 00:57:28.779453 21085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "43e6ae21a7314c1d8c305bebc1f0e7e7"
format_stamp: "Formatted at 2026-08-10 00:57:28 on dist-test-slave-gp6n"
server_key: "26223cf49929c47fcae6bd13669bd15c"
server_key_iv: "45f240b6e3ec2033bb72dd57d493aee8"
server_key_version: "encryptionkey@0"
I20260810 00:57:28.784705 21085 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 00:57:28.788465 21101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:28.790031 21085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 00:57:28.790236 21085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "43e6ae21a7314c1d8c305bebc1f0e7e7"
format_stamp: "Formatted at 2026-08-10 00:57:28 on dist-test-slave-gp6n"
server_key: "26223cf49929c47fcae6bd13669bd15c"
server_key_iv: "45f240b6e3ec2033bb72dd57d493aee8"
server_key_version: "encryptionkey@0"
I20260810 00:57:28.790511 21085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:28.828104 21085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:28.832578 21085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:28.832916 21085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:28.833997 21085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:57:28.835479 21085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:57:28.835593 21085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:28.835769 21085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:57:28.835870 21085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:28.874331 21085 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:39357
I20260810 00:57:28.874367 21230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:39357 every 8 connection(s)
I20260810 00:57:28.876001 21085 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 00:57:28.882423 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 21085
I20260810 00:57:28.882674 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 00:57:28.883154 20806 external_mini_cluster.cc:1428] Setting key 0c0816deb303ee55e0cc97394cb1fb76
Aug 10 00:57:28 dist-test-slave-gp6n krb5kdc[20814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323448, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:28.897938 21233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:28.878592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:57237 (local address 127.20.81.190:41269)
0810 00:57:28.879060 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:28.879064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:28.879979 (+ 915us) server_negotiation.cc:407] Connection header received
0810 00:57:28.881760 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:28.881766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:28.881857 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:28.881984 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:28.883748 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:28.884523 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:28.886054 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:28.886541 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:28.886675 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:28.891104 (+ 4429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:28.891109 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:28.891112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:28.891142 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:28.894102 (+ 2960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:28.895040 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:28.895048 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:28.895054 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:28.895213 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:28.895711 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:28.895719 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:28.895725 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:28.896114 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:28.896280 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:28.897076 (+ 796us) server_negotiation.cc:299] Negotiation successful
0810 00:57:28.897344 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":259,"threads_started":1}
I20260810 00:57:28.898607 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:28.879122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:41269 (local address 127.20.81.130:57237)
0810 00:57:28.879814 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:28.879858 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:28.881499 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:28.882241 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:28.882249 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:28.882959 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:28.883573 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:28.883592 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:28.884677 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:28.884686 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:28.885896 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:28.885908 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:28.886771 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:28.886778 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:28.887009 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:28.887683 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:28.887711 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:28.890880 (+ 3169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:28.894273 (+ 3393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:28.894284 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:28.894301 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:28.894901 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:28.895354 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:28.895360 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:28.895366 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:28.895588 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:28.896279 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:28.896287 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:28.896626 (+ 339us) client_negotiation.cc:769] Sending connection context
0810 00:57:28.896904 (+ 278us) client_negotiation.cc:240] Negotiation successful
0810 00:57:28.897297 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":299,"threads_started":1}
I20260810 00:57:28.900259 21231 heartbeater.cc:347] Connected to a master server at 127.20.81.190:41269
I20260810 00:57:28.900583 21231 heartbeater.cc:464] Registering TS with master...
I20260810 00:57:28.901537 21231 heartbeater.cc:511] Master 127.20.81.190:41269 requested a full tablet report, sending...
I20260810 00:57:28.903283 20865 ts_manager.cc:194] Registered new tserver with Master: 43e6ae21a7314c1d8c305bebc1f0e7e7 (127.20.81.130:39357)
I20260810 00:57:28.904125 20865 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:57237
WARNING: no policy specified for kudu/127.20.81.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
I20260810 00:57:28.934067 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:41269
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:29.271807 21237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:29.272138 21237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:29.272323 21237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:29.279594 21237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:29.279757 21237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:29.279937 21237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 00:57:29.289862 21237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:41269
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:29.291368 21237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:29.292761 21237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:29.307093 21242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:29.307096 21245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:29.307328 21243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:29.308449 21237 server_base.cc:1034] running on GCE node
I20260810 00:57:29.309160 21237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:29.310328 21237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:29.311594 21237 hybrid_clock.cc:648] HybridClock initialized: now 1786323449311533 us; error 46 us; skew 500 ppm
I20260810 00:57:29.311966 21237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:29 dist-test-slave-gp6n krb5kdc[20814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323449, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:29.315696 21237 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 00:57:29.317430 21237 webserver.cc:460] Webserver started at http://127.20.81.131:36611/ using document root <none> and password file <none>
I20260810 00:57:29.318195 21237 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:29.318356 21237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:29.318760 21237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:29.322006 21237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "f6e41b27b93b40a885d4c19579a521ce"
format_stamp: "Formatted at 2026-08-10 00:57:29 on dist-test-slave-gp6n"
server_key: "b20b8bcc84c7d47d2ed42ac23d3840d1"
server_key_iv: "2a3ee6ba8e600e3f76d2dd356600be2f"
server_key_version: "encryptionkey@0"
I20260810 00:57:29.322932 21237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "f6e41b27b93b40a885d4c19579a521ce"
format_stamp: "Formatted at 2026-08-10 00:57:29 on dist-test-slave-gp6n"
server_key: "b20b8bcc84c7d47d2ed42ac23d3840d1"
server_key_iv: "2a3ee6ba8e600e3f76d2dd356600be2f"
server_key_version: "encryptionkey@0"
I20260810 00:57:29.328737 21237 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260810 00:57:29.332324 21252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:29.334059 21237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 00:57:29.334210 21237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "f6e41b27b93b40a885d4c19579a521ce"
format_stamp: "Formatted at 2026-08-10 00:57:29 on dist-test-slave-gp6n"
server_key: "b20b8bcc84c7d47d2ed42ac23d3840d1"
server_key_iv: "2a3ee6ba8e600e3f76d2dd356600be2f"
server_key_version: "encryptionkey@0"
I20260810 00:57:29.334451 21237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:29.364151 21237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:29.368402 21237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:29.368707 21237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:29.369719 21237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:57:29.371083 21237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:57:29.371155 21237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:29.371315 21237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:57:29.371390 21237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:29.371601 21078 heartbeater.cc:503] Master 127.20.81.190:41269 was elected leader, sending a full tablet report...
I20260810 00:57:29.408363 21237 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:37577
I20260810 00:57:29.408413 21384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:37577 every 8 connection(s)
I20260810 00:57:29.409739 21237 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 00:57:29.413576 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 21237
I20260810 00:57:29.413798 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 00:57:29.414386 20806 external_mini_cluster.cc:1428] Setting key 9821a1e6aeedfe5704fe00e817126afb
Aug 10 00:57:29 dist-test-slave-gp6n krb5kdc[20814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323449, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:29.430042 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:29.412381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:52441 (local address 127.20.81.190:41269)
0810 00:57:29.412869 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:29.412874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:29.413879 (+ 1005us) server_negotiation.cc:407] Connection header received
0810 00:57:29.416279 (+ 2400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:29.416286 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:29.416368 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:29.416456 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:29.418366 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:29.419225 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:29.420711 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:29.421127 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:29.421217 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:29.424596 (+ 3379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:29.424601 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:29.424605 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:29.424642 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:29.427155 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:29.427789 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:29.427793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:29.427795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:29.427907 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:29.428267 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:29.428270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:29.428272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:29.428516 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:29.428631 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:29.429421 (+ 790us) server_negotiation.cc:299] Negotiation successful
0810 00:57:29.429649 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":258,"threads_started":1}
I20260810 00:57:29.431138 21388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:29.412832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:41269 (local address 127.20.81.131:52441)
0810 00:57:29.413669 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:29.413730 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:29.416019 (+ 2289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:29.416751 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:29.416763 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:29.417485 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:29.418183 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:29.418201 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:29.419359 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:29.419363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:29.420584 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:29.420596 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:29.421317 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:29.421321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:29.421517 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:29.422205 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:29.422227 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:29.424411 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:29.427273 (+ 2862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:29.427280 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:29.427291 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:29.427694 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:29.428006 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:29.428009 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:29.428011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:29.428179 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:29.428637 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:29.428649 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:29.429036 (+ 387us) client_negotiation.cc:769] Sending connection context
0810 00:57:29.429315 (+ 279us) client_negotiation.cc:240] Negotiation successful
0810 00:57:29.429822 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":315,"threads_started":1}
I20260810 00:57:29.432375 21385 heartbeater.cc:347] Connected to a master server at 127.20.81.190:41269
I20260810 00:57:29.432669 21385 heartbeater.cc:464] Registering TS with master...
I20260810 00:57:29.433357 21385 heartbeater.cc:511] Master 127.20.81.190:41269 requested a full tablet report, sending...
I20260810 00:57:29.434753 20865 ts_manager.cc:194] Registered new tserver with Master: f6e41b27b93b40a885d4c19579a521ce (127.20.81.131:37577)
I20260810 00:57:29.435557 20865 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:52441
I20260810 00:57:29.442751 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 00:57:29 dist-test-slave-gp6n krb5kdc[20814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323449, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 00:57:29 dist-test-slave-gp6n krb5kdc[20814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323449, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.81.129@KRBTEST.COM
I20260810 00:57:29.501734 21393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:29.484192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33828 (local address 127.20.81.129:34453)
0810 00:57:29.484864 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:29.484872 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:29.484899 (+ 27us) server_negotiation.cc:407] Connection header received
0810 00:57:29.484970 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:29.484976 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:29.485187 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:29.485318 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:29.487285 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:29.488431 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:29.490145 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:29.490893 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:29.491002 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:29.495592 (+ 4590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:29.495603 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:29.495619 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:29.495654 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:29.498775 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:29.499531 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:29.499538 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:29.499540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:29.499682 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:29.500146 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:29.500153 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:29.500157 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:29.500643 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:29.500957 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:29.501141 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 00:57:29.501337 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":332,"threads_started":1}
W20260810 00:57:29.503154 20980 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33828
Aug 10 00:57:29 dist-test-slave-gp6n krb5kdc[20814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 00:57:29 dist-test-slave-gp6n krb5kdc[20814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.129@KRBTEST.COM
I20260810 00:57:29.557317 21393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:29.543323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33830 (local address 127.20.81.129:34453)
0810 00:57:29.543568 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:29.543576 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:29.543858 (+ 282us) server_negotiation.cc:407] Connection header received
0810 00:57:29.544083 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:29.544087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:29.544169 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:29.544320 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:29.545667 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:29.546370 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:29.547644 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:29.548086 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:29.548222 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:29.552196 (+ 3974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:29.552200 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:29.552204 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:29.552236 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:29.554770 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:29.555384 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:29.555391 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:29.555394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:29.555523 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:29.555918 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:29.555924 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:29.555927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:29.556200 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:29.556459 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:29.556749 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 00:57:29.556982 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 00:57:29.558926 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 20935
I20260810 00:57:29.571120 21393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:29.560051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33834 (local address 127.20.81.129:34453)
0810 00:57:29.560207 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:29.560211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:29.560521 (+ 310us) server_negotiation.cc:407] Connection header received
0810 00:57:29.560637 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:29.560640 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:29.560691 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:29.560771 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:29.561891 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:29.562628 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:29.563869 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:29.564273 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:29.564363 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:29.566500 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:29.566504 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:29.566507 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:29.566533 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:29.568974 (+ 2441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:29.569559 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:29.569563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:29.569565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:29.569685 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:29.569985 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:29.569987 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:29.569989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:29.570214 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:29.570383 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:29.570689 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 00:57:29.570831 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 00:57:29.571769 21071 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:57:29.656718 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 20935
I20260810 00:57:29.676354 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 21085
Aug 10 00:57:29 dist-test-slave-gp6n krb5kdc[20814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.130@KRBTEST.COM
I20260810 00:57:29.692075 21400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:29.677282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42694 (local address 127.20.81.130:39357)
0810 00:57:29.677814 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:29.677822 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:29.677846 (+ 24us) server_negotiation.cc:407] Connection header received
0810 00:57:29.677906 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:29.677912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:29.678120 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:29.678301 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:29.679551 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:29.680581 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:29.681830 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:29.682383 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:29.682574 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:29.686720 (+ 4146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:29.686731 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:29.686745 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:29.686782 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:29.689526 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:29.690114 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:29.690121 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:29.690123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:29.690246 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:29.690758 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:29.690762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:29.690764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:29.691088 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:29.691294 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:29.691578 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 00:57:29.691770 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":278,"threads_started":1}
I20260810 00:57:29.693307 21226 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:57:29.777268 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 21085
I20260810 00:57:29.798544 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 21237
Aug 10 00:57:29 dist-test-slave-gp6n krb5kdc[20814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.131@KRBTEST.COM
I20260810 00:57:29.815557 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:29.799662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47960 (local address 127.20.81.131:37577)
0810 00:57:29.800336 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:29.800344 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:29.800390 (+ 46us) server_negotiation.cc:407] Connection header received
0810 00:57:29.800455 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:29.800462 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:29.800678 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:29.800802 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:29.802270 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:29.803416 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:29.804676 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:29.805232 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:29.805330 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:29.809861 (+ 4531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:29.809873 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:29.809886 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:29.809929 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:29.812693 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:29.813326 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:29.813332 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:29.813334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:29.813451 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:29.814047 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:29.814050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:29.814052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:29.814364 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:29.814671 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:29.814969 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 00:57:29.815165 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":351,"threads_started":1}
I20260810 00:57:29.816896 21380 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:57:29.900162 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 21237
I20260810 00:57:29.920519 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 20829
I20260810 00:57:29.921921 20899 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:57:30.029407 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 20829
2026-08-10T00:57:30Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4299 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:57:30 dist-test-slave-gp6n krb5kdc[21407](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:57:30 dist-test-slave-gp6n krb5kdc[21407](info): set up 2 sockets
Aug 10 00:57:30 dist-test-slave-gp6n krb5kdc[21407](info): commencing operation
krb5kdc: starting...
W20260810 00:57:31.187485 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.002s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 00:57:31 dist-test-slave-gp6n krb5kdc[21407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:57:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:57:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 00:57:31.391337 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:37299
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:46009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:37299
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:31.716588 21420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:31.716866 21420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:31.717033 21420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:31.724217 21420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:57:31.724321 21420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:31.724400 21420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:31.724485 21420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:57:31.724550 21420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:57:31.735008 21420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:46009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:37299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:37299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:31.736373 21420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:31.737716 21420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:31.749881 21428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:31.749830 21431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:31.749774 21429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:31.750742 21420 server_base.cc:1034] running on GCE node
I20260810 00:57:31.751469 21420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:31.752876 21420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:31.754082 21420 hybrid_clock.cc:648] HybridClock initialized: now 1786323451754036 us; error 31 us; skew 500 ppm
I20260810 00:57:31.754499 21420 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:31 dist-test-slave-gp6n krb5kdc[21407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323451, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:31.757835 21420 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 00:57:31.759428 21420 webserver.cc:460] Webserver started at http://127.20.81.190:44807/ using document root <none> and password file <none>
I20260810 00:57:31.760088 21420 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:31.760195 21420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:31.760557 21420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:31.763536 21420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "34fb28a49d6744598e2bee6bbfed7e19"
format_stamp: "Formatted at 2026-08-10 00:57:31 on dist-test-slave-gp6n"
server_key: "a72d70a3d26e655b31785fc85edb588f"
server_key_iv: "c386af8ad81428b006ee6b68f6c907da"
server_key_version: "encryptionkey@0"
I20260810 00:57:31.764379 21420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "34fb28a49d6744598e2bee6bbfed7e19"
format_stamp: "Formatted at 2026-08-10 00:57:31 on dist-test-slave-gp6n"
server_key: "a72d70a3d26e655b31785fc85edb588f"
server_key_iv: "c386af8ad81428b006ee6b68f6c907da"
server_key_version: "encryptionkey@0"
I20260810 00:57:31.769496 21420 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 00:57:31.772918 21441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:31.774231 21420 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 00:57:31.774453 21420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "34fb28a49d6744598e2bee6bbfed7e19"
format_stamp: "Formatted at 2026-08-10 00:57:31 on dist-test-slave-gp6n"
server_key: "a72d70a3d26e655b31785fc85edb588f"
server_key_iv: "c386af8ad81428b006ee6b68f6c907da"
server_key_version: "encryptionkey@0"
I20260810 00:57:31.774669 21420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:31.797343 21420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:31.801767 21420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:31.802080 21420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:31.821096 21420 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:37299
I20260810 00:57:31.821132 21497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:37299 every 8 connection(s)
I20260810 00:57:31.822597 21420 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:57:31.826299 21498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:31.830510 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 21420
I20260810 00:57:31.830749 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 00:57:31.831416 20806 external_mini_cluster.cc:1428] Setting key 8d075a89f8444f711b5275e274f172a5
I20260810 00:57:31.838546 21498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19: Bootstrap starting.
I20260810 00:57:31.842144 21498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:31.843315 21498 log.cc:824] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:31.846251 21498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19: No bootstrap required, opened a new log
Aug 10 00:57:31 dist-test-slave-gp6n krb5kdc[21407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:31.857084 21501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:31.833455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46860 (local address 127.20.81.190:37299)
0810 00:57:31.834316 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:31.834337 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:31.834388 (+ 51us) server_negotiation.cc:407] Connection header received
0810 00:57:31.836998 (+ 2610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:31.837025 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:31.837505 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:31.837927 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:31.839435 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:31.841490 (+ 2055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:31.842857 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:31.843961 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:31.844134 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:31.848509 (+ 4375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:31.848520 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:31.848538 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:31.848579 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:31.852635 (+ 4056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:31.853302 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:31.853311 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:31.853324 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:31.853529 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:31.853868 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:31.853873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:31.853877 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:31.854603 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:31.854916 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:31.855325 (+ 409us) server_negotiation.cc:299] Negotiation successful
0810 00:57:31.855760 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":360,"threads_started":1}
I20260810 00:57:31.856905 21498 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34fb28a49d6744598e2bee6bbfed7e19" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 37299 } }
I20260810 00:57:31.857359 21498 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:31.857479 21498 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fb28a49d6744598e2bee6bbfed7e19, State: Initialized, Role: FOLLOWER
I20260810 00:57:31.858114 21498 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34fb28a49d6744598e2bee6bbfed7e19" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 37299 } }
I20260810 00:57:31.858448 21498 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:57:31.858590 21498 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:57:31.858790 21498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:31.861207 21498 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34fb28a49d6744598e2bee6bbfed7e19" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 37299 } }
I20260810 00:57:31.861675 21498 leader_election.cc:302] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34fb28a49d6744598e2bee6bbfed7e19; no voters:
I20260810 00:57:31.862229 21498 leader_election.cc:288] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:57:31.862452 21506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:57:31.863199 21506 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19 [term 1 LEADER]: Becoming Leader. State: Replica: 34fb28a49d6744598e2bee6bbfed7e19, State: Running, Role: LEADER
I20260810 00:57:31.863849 21506 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34fb28a49d6744598e2bee6bbfed7e19" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 37299 } }
I20260810 00:57:31.864598 21498 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:57:31.867130 21507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34fb28a49d6744598e2bee6bbfed7e19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34fb28a49d6744598e2bee6bbfed7e19" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 37299 } } }
I20260810 00:57:31.867202 21508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34fb28a49d6744598e2bee6bbfed7e19. Latest consensus state: current_term: 1 leader_uuid: "34fb28a49d6744598e2bee6bbfed7e19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34fb28a49d6744598e2bee6bbfed7e19" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 37299 } } }
I20260810 00:57:31.867568 21507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19 [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:31.867568 21508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19 [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:31.872430 21515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:57:31.877562 21515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:57:31.892481 21515 catalog_manager.cc:1347] Generated new cluster ID: 4db8bd14b3974fc0bdf1a4e9e43f225c
I20260810 00:57:31.892596 21515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:57:31.924854 21515 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:57:31.926288 21515 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:57:31.933756 21515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34fb28a49d6744598e2bee6bbfed7e19: Generated new TSK 0
I20260810 00:57:31.934651 21515 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.81.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
I20260810 00:57:31.992699 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:37299
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:46009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:32.333477 21527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:32.333745 21527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:32.333906 21527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:32.341013 21527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:32.341120 21527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:32.341328 21527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 00:57:32.351109 21527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:46009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:37299
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:32.352478 21527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:32.353832 21527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:32.368547 21533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:32.368548 21534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:32.368683 21537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:32.369510 21527 server_base.cc:1034] running on GCE node
I20260810 00:57:32.370096 21527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:32.371025 21527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:32.372274 21527 hybrid_clock.cc:648] HybridClock initialized: now 1786323452372211 us; error 37 us; skew 500 ppm
I20260810 00:57:32.372565 21527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:32 dist-test-slave-gp6n krb5kdc[21407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323452, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:32.375757 21527 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 00:57:32.377240 21527 webserver.cc:460] Webserver started at http://127.20.81.129:44193/ using document root <none> and password file <none>
I20260810 00:57:32.377882 21527 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:32.377983 21527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:32.378237 21527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:32.381224 21527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "bae3b53b08544e21a2a9024a15138743"
format_stamp: "Formatted at 2026-08-10 00:57:32 on dist-test-slave-gp6n"
server_key: "f28aa348c734706dda9fc5a4d399a870"
server_key_iv: "38e110f3944b96a45d3d065d3063f6dd"
server_key_version: "encryptionkey@0"
I20260810 00:57:32.382019 21527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "bae3b53b08544e21a2a9024a15138743"
format_stamp: "Formatted at 2026-08-10 00:57:32 on dist-test-slave-gp6n"
server_key: "f28aa348c734706dda9fc5a4d399a870"
server_key_iv: "38e110f3944b96a45d3d065d3063f6dd"
server_key_version: "encryptionkey@0"
I20260810 00:57:32.387066 21527 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260810 00:57:32.390751 21545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:32.392051 21527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 00:57:32.392230 21527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "bae3b53b08544e21a2a9024a15138743"
format_stamp: "Formatted at 2026-08-10 00:57:32 on dist-test-slave-gp6n"
server_key: "f28aa348c734706dda9fc5a4d399a870"
server_key_iv: "38e110f3944b96a45d3d065d3063f6dd"
server_key_version: "encryptionkey@0"
I20260810 00:57:32.392406 21527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:32.420432 21527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:32.424472 21527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:32.424763 21527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:32.425770 21527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:57:32.427129 21527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:57:32.427230 21527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:32.427382 21527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:57:32.427464 21527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:32.461858 21527 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:36409
I20260810 00:57:32.461949 21670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:36409 every 8 connection(s)
I20260810 00:57:32.463254 21527 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 00:57:32.463935 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 21527
I20260810 00:57:32.464164 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 00:57:32.465045 20806 external_mini_cluster.cc:1428] Setting key d8a08962ed1e5a47f0b5ef8ef9b3825a
Aug 10 00:57:32 dist-test-slave-gp6n krb5kdc[21407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323452, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.81.130@KRBTEST.COM; defaulting to no policy
I20260810 00:57:32.488185 21673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:32.466256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:39291 (local address 127.20.81.190:37299)
0810 00:57:32.466752 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:32.466757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:32.467875 (+ 1118us) server_negotiation.cc:407] Connection header received
0810 00:57:32.469955 (+ 2080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:32.469963 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:32.470052 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:32.470176 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:32.471668 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:32.472825 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:32.475141 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:32.475533 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:32.475632 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:32.480181 (+ 4549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:32.480190 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:32.480198 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:32.480242 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:32.484300 (+ 4058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:32.485152 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:32.485160 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:32.485164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:32.485325 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:32.485786 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:32.485793 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:32.485797 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:32.486233 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:32.486397 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:32.487630 (+ 1233us) server_negotiation.cc:299] Negotiation successful
0810 00:57:32.487934 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":286,"threads_started":1}
I20260810 00:57:32.489614 21675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:32.466902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:37299 (local address 127.20.81.129:39291)
0810 00:57:32.467714 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:32.467768 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:32.469684 (+ 1916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:32.470366 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:32.470374 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:32.470940 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:32.471503 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:32.471519 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:32.472992 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:32.472995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:32.474999 (+ 2004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:32.475016 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:32.475759 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:32.475768 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:32.476108 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:32.476846 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:32.476880 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:32.479945 (+ 3065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:32.484482 (+ 4537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:32.484490 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:32.484505 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:32.484994 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:32.485445 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:32.485449 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:32.485451 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:32.485646 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:32.486490 (+ 844us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:32.486497 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:32.487070 (+ 573us) client_negotiation.cc:769] Sending connection context
0810 00:57:32.487486 (+ 416us) client_negotiation.cc:240] Negotiation successful
0810 00:57:32.488182 (+ 696us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":338,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.130@KRBTEST.COM" created.
I20260810 00:57:32.491475 21671 heartbeater.cc:347] Connected to a master server at 127.20.81.190:37299
I20260810 00:57:32.491832 21671 heartbeater.cc:464] Registering TS with master...
I20260810 00:57:32.492620 21671 heartbeater.cc:511] Master 127.20.81.190:37299 requested a full tablet report, sending...
I20260810 00:57:32.495123 21460 ts_manager.cc:194] Registered new tserver with Master: bae3b53b08544e21a2a9024a15138743 (127.20.81.129:36409)
I20260810 00:57:32.496737 21460 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:39291
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
I20260810 00:57:32.517984 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:37299
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:46009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:32.852650 21678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:32.852928 21678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:32.853029 21678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:32.860198 21678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:32.860299 21678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:32.860412 21678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 00:57:32.870282 21678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:46009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:37299
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:32.871496 21678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:32.872768 21678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:32.886070 21686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:32.886027 21685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:32.886283 21689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:32.887516 21678 server_base.cc:1034] running on GCE node
I20260810 00:57:32.888053 21678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:32.888964 21678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:32.890178 21678 hybrid_clock.cc:648] HybridClock initialized: now 1786323452890143 us; error 32 us; skew 500 ppm
I20260810 00:57:32.890563 21678 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:32 dist-test-slave-gp6n krb5kdc[21407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323452, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:32.893646 21678 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 00:57:32.895170 21678 webserver.cc:460] Webserver started at http://127.20.81.130:45485/ using document root <none> and password file <none>
I20260810 00:57:32.895843 21678 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:32.896023 21678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:32.896365 21678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:32.899427 21678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "cf8b096d2f0345ea8d8fbf11dd6cfb51"
format_stamp: "Formatted at 2026-08-10 00:57:32 on dist-test-slave-gp6n"
server_key: "915c98ab7de3bd73e45a2f4db9f1f1c7"
server_key_iv: "3d77d21942608254cac14e2a5dc19d50"
server_key_version: "encryptionkey@0"
I20260810 00:57:32.900221 21678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "cf8b096d2f0345ea8d8fbf11dd6cfb51"
format_stamp: "Formatted at 2026-08-10 00:57:32 on dist-test-slave-gp6n"
server_key: "915c98ab7de3bd73e45a2f4db9f1f1c7"
server_key_iv: "3d77d21942608254cac14e2a5dc19d50"
server_key_version: "encryptionkey@0"
I20260810 00:57:32.905282 21678 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 00:57:32.908593 21697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:32.909857 21678 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260810 00:57:32.910020 21678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "cf8b096d2f0345ea8d8fbf11dd6cfb51"
format_stamp: "Formatted at 2026-08-10 00:57:32 on dist-test-slave-gp6n"
server_key: "915c98ab7de3bd73e45a2f4db9f1f1c7"
server_key_iv: "3d77d21942608254cac14e2a5dc19d50"
server_key_version: "encryptionkey@0"
I20260810 00:57:32.910213 21678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:32.923303 21678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:32.927126 21678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:32.927453 21678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:32.928391 21678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:57:32.929718 21678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:57:32.929792 21678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:32.929973 21678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:57:32.930030 21678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:32.963838 21678 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:43491
I20260810 00:57:32.963927 21826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:43491 every 8 connection(s)
I20260810 00:57:32.965222 21678 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 00:57:32.966710 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 21678
I20260810 00:57:32.966934 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 00:57:32.967511 20806 external_mini_cluster.cc:1428] Setting key bb76b28157c99759ce70056793dbdbed
Aug 10 00:57:32 dist-test-slave-gp6n krb5kdc[21407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323452, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:32.990125 21673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:32.968197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:52143 (local address 127.20.81.190:37299)
0810 00:57:32.968421 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:32.968510 (+ 89us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:32.969906 (+ 1396us) server_negotiation.cc:407] Connection header received
0810 00:57:32.972894 (+ 2988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:32.972898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:32.972971 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:32.973063 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:32.975194 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:32.975975 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:32.977902 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:32.978771 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:32.978911 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:32.983225 (+ 4314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:32.983230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:32.983234 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:32.983269 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:32.986587 (+ 3318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:32.987633 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:32.987639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:32.987643 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:32.987793 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:32.988268 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:32.988273 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:32.988277 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:32.988577 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:32.988745 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:32.989498 (+ 753us) server_negotiation.cc:299] Negotiation successful
0810 00:57:32.989739 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 00:57:32.991350 21829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:32.968823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:37299 (local address 127.20.81.130:52143)
0810 00:57:32.969728 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:32.969783 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:32.972575 (+ 2792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:32.973329 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:32.973343 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:32.974134 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:32.975004 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:32.975031 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:32.976128 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:32.976136 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:32.977761 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:32.977773 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:32.979004 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:32.979014 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:32.979356 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:32.980113 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:32.980146 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:32.982982 (+ 2836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:32.986797 (+ 3815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:32.986810 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:32.986829 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:32.987495 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:32.987924 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:32.987930 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:32.987934 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:32.988137 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:32.988720 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:32.988727 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:32.989083 (+ 356us) client_negotiation.cc:769] Sending connection context
0810 00:57:32.989385 (+ 302us) client_negotiation.cc:240] Negotiation successful
0810 00:57:32.989959 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":383,"threads_started":1}
WARNING: no policy specified for kudu/127.20.81.131@KRBTEST.COM; defaulting to no policy
I20260810 00:57:32.993201 21827 heartbeater.cc:347] Connected to a master server at 127.20.81.190:37299
I20260810 00:57:32.993557 21827 heartbeater.cc:464] Registering TS with master...
I20260810 00:57:32.994618 21827 heartbeater.cc:511] Master 127.20.81.190:37299 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.131@KRBTEST.COM" created.
I20260810 00:57:32.996428 21460 ts_manager.cc:194] Registered new tserver with Master: cf8b096d2f0345ea8d8fbf11dd6cfb51 (127.20.81.130:43491)
I20260810 00:57:32.997601 21460 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:52143
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
I20260810 00:57:33.021533 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:37299
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:46009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:33.350972 21833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:33.351227 21833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:33.351387 21833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:33.358633 21833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:33.358754 21833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:33.358946 21833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 00:57:33.370224 21833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:46009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:37299
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:33.371541 21833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:33.372920 21833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:33.386140 21838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:33.386083 21839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:33.386060 21841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:33.387197 21833 server_base.cc:1034] running on GCE node
I20260810 00:57:33.387850 21833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:33.388875 21833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:33.390125 21833 hybrid_clock.cc:648] HybridClock initialized: now 1786323453390083 us; error 56 us; skew 500 ppm
I20260810 00:57:33.390507 21833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:33 dist-test-slave-gp6n krb5kdc[21407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323453, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:33.393850 21833 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 00:57:33.395381 21833 webserver.cc:460] Webserver started at http://127.20.81.131:45235/ using document root <none> and password file <none>
I20260810 00:57:33.396068 21833 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:33.396198 21833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:33.396559 21833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:33.399542 21833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "cc437866272647ab84661fbf340925f2"
format_stamp: "Formatted at 2026-08-10 00:57:33 on dist-test-slave-gp6n"
server_key: "1e64e6abca6488bf2807d6a5929b2117"
server_key_iv: "eef1020bd9691af3604a2b148515541e"
server_key_version: "encryptionkey@0"
I20260810 00:57:33.400355 21833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "cc437866272647ab84661fbf340925f2"
format_stamp: "Formatted at 2026-08-10 00:57:33 on dist-test-slave-gp6n"
server_key: "1e64e6abca6488bf2807d6a5929b2117"
server_key_iv: "eef1020bd9691af3604a2b148515541e"
server_key_version: "encryptionkey@0"
I20260810 00:57:33.405517 21833 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 00:57:33.409149 21849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:33.410585 21833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 00:57:33.410744 21833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "cc437866272647ab84661fbf340925f2"
format_stamp: "Formatted at 2026-08-10 00:57:33 on dist-test-slave-gp6n"
server_key: "1e64e6abca6488bf2807d6a5929b2117"
server_key_iv: "eef1020bd9691af3604a2b148515541e"
server_key_version: "encryptionkey@0"
I20260810 00:57:33.410930 21833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:33.424300 21833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:33.428650 21833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:33.428939 21833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:33.429908 21833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:57:33.431300 21833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:57:33.431375 21833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:33.431488 21833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:57:33.431560 21833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:33.465721 21833 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:39849
I20260810 00:57:33.465768 21980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:39849 every 8 connection(s)
I20260810 00:57:33.467136 21833 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 00:57:33.469462 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 21833
I20260810 00:57:33.469714 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 00:57:33.470213 20806 external_mini_cluster.cc:1428] Setting key 344ecc81e04ea295022dfc8fb8b10b3d
Aug 10 00:57:33 dist-test-slave-gp6n krb5kdc[21407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323453, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:33.487443 21673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:33.469886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:49543 (local address 127.20.81.190:37299)
0810 00:57:33.470093 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:33.470099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:33.471424 (+ 1325us) server_negotiation.cc:407] Connection header received
0810 00:57:33.473311 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:33.473317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:33.473402 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:33.473523 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:33.475029 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.475783 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.477310 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.477772 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.477906 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:33.481487 (+ 3581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:33.481492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:33.481495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:33.481528 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:33.484392 (+ 2864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:33.485114 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:33.485126 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:33.485129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:33.485259 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:33.485698 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:33.485707 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:33.485709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:33.485969 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:33.486125 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:33.486886 (+ 761us) server_negotiation.cc:299] Negotiation successful
0810 00:57:33.487098 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 00:57:33.488350 21984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:33.470380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:37299 (local address 127.20.81.131:49543)
0810 00:57:33.471261 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:33.471316 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:33.473033 (+ 1717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:33.473749 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:33.473757 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:33.474249 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:33.474862 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:33.474879 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.475942 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.475946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:33.477160 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:33.477171 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.478006 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.478013 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:33.478307 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:33.479118 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:33.479140 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:33.481303 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:33.484546 (+ 3243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:33.484554 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:33.484569 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:33.484986 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:33.485374 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:33.485379 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:33.485382 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:33.485575 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:33.486096 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:33.486103 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:33.486497 (+ 394us) client_negotiation.cc:769] Sending connection context
0810 00:57:33.486763 (+ 266us) client_negotiation.cc:240] Negotiation successful
0810 00:57:33.487281 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":341,"threads_started":1}
I20260810 00:57:33.489621 21981 heartbeater.cc:347] Connected to a master server at 127.20.81.190:37299
I20260810 00:57:33.489926 21981 heartbeater.cc:464] Registering TS with master...
I20260810 00:57:33.490700 21981 heartbeater.cc:511] Master 127.20.81.190:37299 requested a full tablet report, sending...
I20260810 00:57:33.492100 21460 ts_manager.cc:194] Registered new tserver with Master: cc437866272647ab84661fbf340925f2 (127.20.81.131:39849)
I20260810 00:57:33.492879 21460 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:49543
I20260810 00:57:33.497491 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:57:33.500471 21671 heartbeater.cc:503] Master 127.20.81.190:37299 was elected leader, sending a full tablet report...
I20260810 00:57:33.517473 21673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:33.504098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46862 (local address 127.20.81.190:37299)
0810 00:57:33.504302 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:33.504308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:33.504917 (+ 609us) server_negotiation.cc:407] Connection header received
0810 00:57:33.504978 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:33.504982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:33.505039 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:33.505136 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:33.506661 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.507445 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.508714 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.509117 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.509212 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:33.511511 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:33.511517 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:33.511521 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:33.511559 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:33.514809 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:33.515425 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:33.515430 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:33.515432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:33.515566 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:33.515946 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:33.515949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:33.515952 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:33.516306 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:33.516506 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:33.516796 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 00:57:33.516961 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 00:57:33.526260 21460 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:57:33.536132 21460 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:57:33.566951 21995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:33.551815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:36409 (local address 127.0.0.1:44858)
0810 00:57:33.552485 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:33.552533 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:33.552708 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:33.558686 (+ 5978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:33.558690 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:33.558713 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:57:33.559049 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:33.559059 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.560967 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.560973 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:33.563575 (+ 2602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:33.563589 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.565825 (+ 2236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.565831 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:33.566024 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:33.566076 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 00:57:33.566214 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 00:57:33.566454 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":29,"spinlock_wait_cycles":3968,"thread_start_us":539,"threads_started":1}
I20260810 00:57:33.567320 21999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:33.555690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50170 (local address 127.20.81.131:39849)
0810 00:57:33.558443 (+ 2753us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:33.558456 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:33.558487 (+ 31us) server_negotiation.cc:407] Connection header received
0810 00:57:33.558578 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:33.558585 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:33.558769 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:33.558940 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:57:33.560496 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.562156 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.564623 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.566049 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.566192 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:33.566467 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:33.566799 (+ 332us) server_negotiation.cc:299] Negotiation successful
0810 00:57:33.566991 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2562,"thread_start_us":2386,"threads_started":1}
I20260810 00:57:33.567826 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:33.552258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44858 (local address 127.20.81.129:36409)
0810 00:57:33.558106 (+ 5848us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:33.558115 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:33.558141 (+ 26us) server_negotiation.cc:407] Connection header received
0810 00:57:33.558228 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:33.558235 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:33.558505 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:33.558687 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:57:33.559199 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.560799 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.563765 (+ 2966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.565322 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.565465 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:33.565709 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:33.567200 (+ 1491us) server_negotiation.cc:299] Negotiation successful
0810 00:57:33.567371 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5633,"thread_start_us":5456,"threads_started":1}
I20260810 00:57:33.567991 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:33.555319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:39849 (local address 127.0.0.1:50170)
0810 00:57:33.556017 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:33.556047 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:33.556160 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:33.559853 (+ 3693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:33.559858 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:33.559902 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:57:33.560293 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:33.560304 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.562281 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.562286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:33.564456 (+ 2170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:33.564471 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.566271 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.566276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:33.566531 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:33.566643 (+ 112us) client_negotiation.cc:769] Sending connection context
0810 00:57:33.566735 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 00:57:33.566954 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"mutex_wait_us":123,"thread_start_us":328,"threads_started":1}
I20260810 00:57:33.568253 21996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:33.552226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48354 (local address 127.20.81.130:43491)
0810 00:57:33.556600 (+ 4374us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:33.556613 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:33.556650 (+ 37us) server_negotiation.cc:407] Connection header received
0810 00:57:33.556768 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:33.556780 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:33.557086 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:33.557326 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:57:33.558080 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.559715 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.564052 (+ 4337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.565675 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.565830 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:33.566065 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:33.567700 (+ 1635us) server_negotiation.cc:299] Negotiation successful
0810 00:57:33.567942 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4197,"thread_start_us":4093,"threads_started":1}
I20260810 00:57:33.570670 21994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:33.551664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:43491 (local address 127.0.0.1:48354)
0810 00:57:33.552486 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:33.552533 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:33.552707 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:33.557351 (+ 4644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:33.557362 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:33.557401 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:57:33.557854 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:33.557882 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.560545 (+ 2663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.560553 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:33.563862 (+ 3309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:33.563876 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.566255 (+ 2379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.566263 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:33.566529 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:33.566644 (+ 115us) client_negotiation.cc:769] Sending connection context
0810 00:57:33.566735 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 00:57:33.566954 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"mutex_wait_us":225,"spinlock_wait_cycles":13952,"thread_start_us":753,"threads_started":1}
I20260810 00:57:33.572049 21907 tablet_service.cc:1467] Processing CreateTablet for tablet 6d1ecb6233934bada07cdbc222d0d4e2 (DEFAULT_TABLE table=test-table [id=bc3e49d8ad4f44dd829980912eb63d1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:57:33.572062 21599 tablet_service.cc:1467] Processing CreateTablet for tablet 6d1ecb6233934bada07cdbc222d0d4e2 (DEFAULT_TABLE table=test-table [id=bc3e49d8ad4f44dd829980912eb63d1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:57:33.573613 21599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d1ecb6233934bada07cdbc222d0d4e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:33.573613 21907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d1ecb6233934bada07cdbc222d0d4e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:33.580611 21754 tablet_service.cc:1467] Processing CreateTablet for tablet 6d1ecb6233934bada07cdbc222d0d4e2 (DEFAULT_TABLE table=test-table [id=bc3e49d8ad4f44dd829980912eb63d1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:57:33.583462 21754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d1ecb6233934bada07cdbc222d0d4e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:33.591969 22003 tablet_bootstrap.cc:492] T 6d1ecb6233934bada07cdbc222d0d4e2 P cc437866272647ab84661fbf340925f2: Bootstrap starting.
I20260810 00:57:33.595170 22005 tablet_bootstrap.cc:492] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743: Bootstrap starting.
I20260810 00:57:33.595742 22003 tablet_bootstrap.cc:654] T 6d1ecb6233934bada07cdbc222d0d4e2 P cc437866272647ab84661fbf340925f2: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:33.596983 22006 tablet_bootstrap.cc:492] T 6d1ecb6233934bada07cdbc222d0d4e2 P cf8b096d2f0345ea8d8fbf11dd6cfb51: Bootstrap starting.
I20260810 00:57:33.597091 22003 log.cc:824] T 6d1ecb6233934bada07cdbc222d0d4e2 P cc437866272647ab84661fbf340925f2: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:33.599050 22005 tablet_bootstrap.cc:654] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:33.600023 22003 tablet_bootstrap.cc:492] T 6d1ecb6233934bada07cdbc222d0d4e2 P cc437866272647ab84661fbf340925f2: No bootstrap required, opened a new log
I20260810 00:57:33.600289 22003 ts_tablet_manager.cc:1397] T 6d1ecb6233934bada07cdbc222d0d4e2 P cc437866272647ab84661fbf340925f2: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 00:57:33.600337 22005 log.cc:824] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:33.600414 22006 tablet_bootstrap.cc:654] T 6d1ecb6233934bada07cdbc222d0d4e2 P cf8b096d2f0345ea8d8fbf11dd6cfb51: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:33.601866 22006 log.cc:824] T 6d1ecb6233934bada07cdbc222d0d4e2 P cf8b096d2f0345ea8d8fbf11dd6cfb51: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:33.605399 22006 tablet_bootstrap.cc:492] T 6d1ecb6233934bada07cdbc222d0d4e2 P cf8b096d2f0345ea8d8fbf11dd6cfb51: No bootstrap required, opened a new log
I20260810 00:57:33.605712 22006 ts_tablet_manager.cc:1397] T 6d1ecb6233934bada07cdbc222d0d4e2 P cf8b096d2f0345ea8d8fbf11dd6cfb51: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 00:57:33.608076 22005 tablet_bootstrap.cc:492] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743: No bootstrap required, opened a new log
I20260810 00:57:33.608330 22005 ts_tablet_manager.cc:1397] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20260810 00:57:33.608666 22003 raft_consensus.cc:348] T 6d1ecb6233934bada07cdbc222d0d4e2 P cc437866272647ab84661fbf340925f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8b096d2f0345ea8d8fbf11dd6cfb51" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 43491 } } peers { permanent_uuid: "bae3b53b08544e21a2a9024a15138743" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 36409 } } peers { permanent_uuid: "cc437866272647ab84661fbf340925f2" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39849 } }
I20260810 00:57:33.609010 22003 raft_consensus.cc:374] T 6d1ecb6233934bada07cdbc222d0d4e2 P cc437866272647ab84661fbf340925f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:33.609102 22003 raft_consensus.cc:729] T 6d1ecb6233934bada07cdbc222d0d4e2 P cc437866272647ab84661fbf340925f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc437866272647ab84661fbf340925f2, State: Initialized, Role: FOLLOWER
I20260810 00:57:33.609685 22003 consensus_queue.cc:260] T 6d1ecb6233934bada07cdbc222d0d4e2 P cc437866272647ab84661fbf340925f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8b096d2f0345ea8d8fbf11dd6cfb51" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 43491 } } peers { permanent_uuid: "bae3b53b08544e21a2a9024a15138743" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 36409 } } peers { permanent_uuid: "cc437866272647ab84661fbf340925f2" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39849 } }
I20260810 00:57:33.610767 21981 heartbeater.cc:503] Master 127.20.81.190:37299 was elected leader, sending a full tablet report...
I20260810 00:57:33.611941 22003 ts_tablet_manager.cc:1428] T 6d1ecb6233934bada07cdbc222d0d4e2 P cc437866272647ab84661fbf340925f2: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260810 00:57:33.616128 22006 raft_consensus.cc:348] T 6d1ecb6233934bada07cdbc222d0d4e2 P cf8b096d2f0345ea8d8fbf11dd6cfb51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8b096d2f0345ea8d8fbf11dd6cfb51" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 43491 } } peers { permanent_uuid: "bae3b53b08544e21a2a9024a15138743" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 36409 } } peers { permanent_uuid: "cc437866272647ab84661fbf340925f2" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39849 } }
I20260810 00:57:33.616568 22006 raft_consensus.cc:374] T 6d1ecb6233934bada07cdbc222d0d4e2 P cf8b096d2f0345ea8d8fbf11dd6cfb51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:33.616662 22006 raft_consensus.cc:729] T 6d1ecb6233934bada07cdbc222d0d4e2 P cf8b096d2f0345ea8d8fbf11dd6cfb51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf8b096d2f0345ea8d8fbf11dd6cfb51, State: Initialized, Role: FOLLOWER
I20260810 00:57:33.617251 22006 consensus_queue.cc:260] T 6d1ecb6233934bada07cdbc222d0d4e2 P cf8b096d2f0345ea8d8fbf11dd6cfb51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8b096d2f0345ea8d8fbf11dd6cfb51" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 43491 } } peers { permanent_uuid: "bae3b53b08544e21a2a9024a15138743" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 36409 } } peers { permanent_uuid: "cc437866272647ab84661fbf340925f2" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39849 } }
I20260810 00:57:33.618436 21827 heartbeater.cc:503] Master 127.20.81.190:37299 was elected leader, sending a full tablet report...
I20260810 00:57:33.619030 22006 ts_tablet_manager.cc:1428] T 6d1ecb6233934bada07cdbc222d0d4e2 P cf8b096d2f0345ea8d8fbf11dd6cfb51: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260810 00:57:33.619293 22005 raft_consensus.cc:348] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8b096d2f0345ea8d8fbf11dd6cfb51" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 43491 } } peers { permanent_uuid: "bae3b53b08544e21a2a9024a15138743" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 36409 } } peers { permanent_uuid: "cc437866272647ab84661fbf340925f2" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39849 } }
I20260810 00:57:33.619649 22005 raft_consensus.cc:374] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:33.619766 22005 raft_consensus.cc:729] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bae3b53b08544e21a2a9024a15138743, State: Initialized, Role: FOLLOWER
I20260810 00:57:33.620314 22005 consensus_queue.cc:260] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8b096d2f0345ea8d8fbf11dd6cfb51" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 43491 } } peers { permanent_uuid: "bae3b53b08544e21a2a9024a15138743" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 36409 } } peers { permanent_uuid: "cc437866272647ab84661fbf340925f2" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39849 } }
I20260810 00:57:33.621975 22005 ts_tablet_manager.cc:1428] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
W20260810 00:57:33.716325 21672 tablet.cc:2369] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:57:33.717484 21828 tablet.cc:2369] T 6d1ecb6233934bada07cdbc222d0d4e2 P cf8b096d2f0345ea8d8fbf11dd6cfb51: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:57:33.718848 21982 tablet.cc:2369] T 6d1ecb6233934bada07cdbc222d0d4e2 P cc437866272647ab84661fbf340925f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:57:33.789973 22012 raft_consensus.cc:482] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:57:33.790302 22012 raft_consensus.cc:504] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8b096d2f0345ea8d8fbf11dd6cfb51" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 43491 } } peers { permanent_uuid: "bae3b53b08544e21a2a9024a15138743" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 36409 } } peers { permanent_uuid: "cc437866272647ab84661fbf340925f2" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39849 } }
I20260810 00:57:33.792107 22012 leader_election.cc:288] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf8b096d2f0345ea8d8fbf11dd6cfb51 (127.20.81.130:43491), cc437866272647ab84661fbf340925f2 (127.20.81.131:39849)
I20260810 00:57:33.798622 21999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:33.792573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:51179 (local address 127.20.81.131:39849)
0810 00:57:33.792772 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:33.792778 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:33.793326 (+ 548us) server_negotiation.cc:407] Connection header received
0810 00:57:33.793575 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:33.793579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:33.793643 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:33.793744 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:57:33.794276 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.795085 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.796899 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.797670 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.797855 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:33.797941 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:33.798205 (+ 264us) server_negotiation.cc:299] Negotiation successful
0810 00:57:33.798325 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 00:57:33.798738 22013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:33.792454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:39849 (local address 127.20.81.129:51179)
0810 00:57:33.793207 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:33.793223 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:33.793406 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:33.793764 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:33.793768 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:33.793804 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:57:33.794132 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:33.794141 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.795216 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.795223 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:33.796754 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:33.796765 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.797885 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.797891 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:33.798044 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:33.798068 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 00:57:33.798169 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 00:57:33.798288 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":475,"threads_started":1}
I20260810 00:57:33.799391 21996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:33.792573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:36987 (local address 127.20.81.130:43491)
0810 00:57:33.792771 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:33.792778 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:33.793751 (+ 973us) server_negotiation.cc:407] Connection header received
0810 00:57:33.794018 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:33.794022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:33.794081 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:33.794177 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:57:33.794809 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.795563 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.797543 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.798300 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.798493 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:33.798573 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:33.798993 (+ 420us) server_negotiation.cc:299] Negotiation successful
0810 00:57:33.799128 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 00:57:33.799428 22014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:33.792592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:43491 (local address 127.20.81.129:36987)
0810 00:57:33.793630 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:33.793644 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:33.793853 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:33.794192 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:33.794196 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:33.794218 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:57:33.794635 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:33.794644 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.795737 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.795742 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:33.797397 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:33.797408 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.798550 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.798556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:33.798735 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:33.798780 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 00:57:33.798913 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 00:57:33.799046 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":939,"mutex_wait_us":19,"spinlock_wait_cycles":23936,"thread_start_us":759,"threads_started":1}
I20260810 00:57:33.799768 21928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d1ecb6233934bada07cdbc222d0d4e2" candidate_uuid: "bae3b53b08544e21a2a9024a15138743" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc437866272647ab84661fbf340925f2" is_pre_election: true
I20260810 00:57:33.800369 21928 raft_consensus.cc:2454] T 6d1ecb6233934bada07cdbc222d0d4e2 P cc437866272647ab84661fbf340925f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bae3b53b08544e21a2a9024a15138743 in term 0.
I20260810 00:57:33.800491 21777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d1ecb6233934bada07cdbc222d0d4e2" candidate_uuid: "bae3b53b08544e21a2a9024a15138743" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf8b096d2f0345ea8d8fbf11dd6cfb51" is_pre_election: true
I20260810 00:57:33.801038 21777 raft_consensus.cc:2454] T 6d1ecb6233934bada07cdbc222d0d4e2 P cf8b096d2f0345ea8d8fbf11dd6cfb51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bae3b53b08544e21a2a9024a15138743 in term 0.
I20260810 00:57:33.801100 21548 leader_election.cc:302] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bae3b53b08544e21a2a9024a15138743, cc437866272647ab84661fbf340925f2; no voters:
I20260810 00:57:33.801543 22012 raft_consensus.cc:2781] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:57:33.801718 22012 raft_consensus.cc:482] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:57:33.801851 22012 raft_consensus.cc:3037] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:33.804145 22012 raft_consensus.cc:504] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8b096d2f0345ea8d8fbf11dd6cfb51" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 43491 } } peers { permanent_uuid: "bae3b53b08544e21a2a9024a15138743" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 36409 } } peers { permanent_uuid: "cc437866272647ab84661fbf340925f2" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39849 } }
I20260810 00:57:33.804698 22012 leader_election.cc:288] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743 [CANDIDATE]: Term 1 election: Requested vote from peers cf8b096d2f0345ea8d8fbf11dd6cfb51 (127.20.81.130:43491), cc437866272647ab84661fbf340925f2 (127.20.81.131:39849)
I20260810 00:57:33.805163 21928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d1ecb6233934bada07cdbc222d0d4e2" candidate_uuid: "bae3b53b08544e21a2a9024a15138743" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc437866272647ab84661fbf340925f2"
I20260810 00:57:33.805203 21777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d1ecb6233934bada07cdbc222d0d4e2" candidate_uuid: "bae3b53b08544e21a2a9024a15138743" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf8b096d2f0345ea8d8fbf11dd6cfb51"
I20260810 00:57:33.805359 21928 raft_consensus.cc:3037] T 6d1ecb6233934bada07cdbc222d0d4e2 P cc437866272647ab84661fbf340925f2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:33.805411 21777 raft_consensus.cc:3037] T 6d1ecb6233934bada07cdbc222d0d4e2 P cf8b096d2f0345ea8d8fbf11dd6cfb51 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:33.807716 21777 raft_consensus.cc:2454] T 6d1ecb6233934bada07cdbc222d0d4e2 P cf8b096d2f0345ea8d8fbf11dd6cfb51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bae3b53b08544e21a2a9024a15138743 in term 1.
I20260810 00:57:33.808058 21928 raft_consensus.cc:2454] T 6d1ecb6233934bada07cdbc222d0d4e2 P cc437866272647ab84661fbf340925f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bae3b53b08544e21a2a9024a15138743 in term 1.
I20260810 00:57:33.808158 21551 leader_election.cc:302] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bae3b53b08544e21a2a9024a15138743, cf8b096d2f0345ea8d8fbf11dd6cfb51; no voters:
I20260810 00:57:33.808483 22012 raft_consensus.cc:2781] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:57:33.808975 22012 raft_consensus.cc:686] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743 [term 1 LEADER]: Becoming Leader. State: Replica: bae3b53b08544e21a2a9024a15138743, State: Running, Role: LEADER
I20260810 00:57:33.809455 22012 consensus_queue.cc:237] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8b096d2f0345ea8d8fbf11dd6cfb51" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 43491 } } peers { permanent_uuid: "bae3b53b08544e21a2a9024a15138743" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 36409 } } peers { permanent_uuid: "cc437866272647ab84661fbf340925f2" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39849 } }
I20260810 00:57:33.814298 21460 catalog_manager.cc:5495] T 6d1ecb6233934bada07cdbc222d0d4e2 P bae3b53b08544e21a2a9024a15138743 reported cstate change: term changed from 0 to 1, leader changed from <none> to bae3b53b08544e21a2a9024a15138743 (127.20.81.129). New cstate: current_term: 1 leader_uuid: "bae3b53b08544e21a2a9024a15138743" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8b096d2f0345ea8d8fbf11dd6cfb51" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 43491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bae3b53b08544e21a2a9024a15138743" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 36409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc437866272647ab84661fbf340925f2" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39849 } health_report { overall_health: UNKNOWN } } }
Aug 10 00:57:33 dist-test-slave-gp6n krb5kdc[21407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.129@KRBTEST.COM
I20260810 00:57:33.867443 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:33.847629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44866 (local address 127.20.81.129:36409)
0810 00:57:33.847860 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:33.847869 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:33.848244 (+ 375us) server_negotiation.cc:407] Connection header received
0810 00:57:33.848466 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:33.848470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:33.848552 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:33.848663 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:33.850247 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.851046 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.852349 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.852764 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.852900 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:33.861408 (+ 8508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:33.861425 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:33.861441 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:33.861480 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:33.864400 (+ 2920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:33.865265 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:33.865273 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:33.865275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:33.865453 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:33.865836 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:33.865841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:33.865844 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:33.866223 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:33.866564 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:33.866854 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 00:57:33.867084 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Aug 10 00:57:33 dist-test-slave-gp6n krb5kdc[21407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323453, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 00:57:33 dist-test-slave-gp6n krb5kdc[21407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323453, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.81.129@KRBTEST.COM
I20260810 00:57:33.917949 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:33.903984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44870 (local address 127.20.81.129:36409)
0810 00:57:33.904252 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:33.904258 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:33.904534 (+ 276us) server_negotiation.cc:407] Connection header received
0810 00:57:33.904683 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:33.904687 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:33.904755 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:33.904842 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:33.906229 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.907033 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.908303 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.908707 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.908797 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:33.912562 (+ 3765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:33.912568 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:33.912571 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:33.912604 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:33.915516 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:33.916145 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:33.916150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:33.916152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:33.916295 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:33.916685 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:33.916689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:33.916692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:33.916975 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:33.917197 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:33.917412 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 00:57:33.917573 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
Aug 10 00:57:33 dist-test-slave-gp6n krb5kdc[21407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 00:57:33 dist-test-slave-gp6n krb5kdc[21407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.129@KRBTEST.COM
I20260810 00:57:33.971047 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:33.956105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44886 (local address 127.20.81.129:36409)
0810 00:57:33.956409 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:33.956416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:33.956619 (+ 203us) server_negotiation.cc:407] Connection header received
0810 00:57:33.956763 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:33.956768 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:33.956857 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:33.956943 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:33.958305 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.959143 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.960427 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.960880 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.960999 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:33.965236 (+ 4237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:33.965241 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:33.965245 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:33.965279 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:33.968099 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:33.968900 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:33.968907 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:33.968912 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:33.969059 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:33.969511 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:33.969516 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:33.969518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:33.969853 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:33.970116 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:33.970455 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 00:57:33.970698 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
W20260810 00:57:33.972736 21576 tablet_service.cc:2908] Rejecting scan request for tablet 6d1ecb6233934bada07cdbc222d0d4e2: Uninitialized: clean time has not yet been initialized
I20260810 00:57:33.976022 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 21527
I20260810 00:57:33.990242 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:33.977468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44890 (local address 127.20.81.129:36409)
0810 00:57:33.977636 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:33.977640 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:33.978924 (+ 1284us) server_negotiation.cc:407] Connection header received
0810 00:57:33.979052 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:33.979055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:33.979111 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:33.979187 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:33.980302 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.981020 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.982188 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:33.982672 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:33.982772 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:33.985440 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:33.985445 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:33.985448 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:33.985478 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:33.987971 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:33.988546 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:33.988550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:33.988552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:33.988663 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:33.989004 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:33.989012 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:33.989018 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:33.989290 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:33.989476 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:33.989777 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 00:57:33.989993 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 00:57:33.990969 21666 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:57:34.083455 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 21527
I20260810 00:57:34.104508 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 21678
Aug 10 00:57:34 dist-test-slave-gp6n krb5kdc[21407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.130@KRBTEST.COM
I20260810 00:57:34.119514 21996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:34.105381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48370 (local address 127.20.81.130:43491)
0810 00:57:34.105557 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:34.105562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:34.105581 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:57:34.105644 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:34.105647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:34.105713 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:34.105810 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:34.107079 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:34.107934 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:34.109243 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:34.109675 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:34.109800 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:34.113652 (+ 3852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:34.113664 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:34.113679 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:34.113715 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:34.116845 (+ 3130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:34.117461 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:34.117468 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:34.117471 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:34.117589 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:34.117985 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:34.117988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:34.117990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:34.118367 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:34.118691 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:34.119021 (+ 330us) server_negotiation.cc:299] Negotiation successful
0810 00:57:34.119176 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 00:57:34.120258 21822 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:57:34.217698 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 21678
I20260810 00:57:34.238799 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 21833
Aug 10 00:57:34 dist-test-slave-gp6n krb5kdc[21407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.131@KRBTEST.COM
I20260810 00:57:34.254102 21999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:34.239821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50180 (local address 127.20.81.131:39849)
0810 00:57:34.240046 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:34.240052 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:34.240075 (+ 23us) server_negotiation.cc:407] Connection header received
0810 00:57:34.240262 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:34.240275 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:34.240340 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:34.240447 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:34.241717 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:34.242532 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:34.243835 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:34.244243 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:34.244400 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:34.248830 (+ 4430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:34.248844 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:34.248860 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:34.248893 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:34.251602 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:34.252270 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:34.252277 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:34.252279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:34.252411 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:34.252763 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:34.252766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:34.252768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:34.253131 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:34.253340 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:34.253667 (+ 327us) server_negotiation.cc:299] Negotiation successful
0810 00:57:34.253836 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 00:57:34.254863 21976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:57:34.347657 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 21833
I20260810 00:57:34.366698 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 21420
I20260810 00:57:34.367868 21493 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:57:34.478576 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 21420
2026-08-10T00:57:34Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4447 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:57:34 dist-test-slave-gp6n krb5kdc[22039](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:57:34 dist-test-slave-gp6n krb5kdc[22039](info): set up 2 sockets
Aug 10 00:57:34 dist-test-slave-gp6n krb5kdc[22039](info): commencing operation
krb5kdc: starting...
W20260810 00:57:35.648348 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.003s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 00:57:35 dist-test-slave-gp6n krb5kdc[22039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:57:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:57:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 00:57:35.862323 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:36433
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:36923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:36433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:36.190023 22054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:36.190310 22054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:36.190461 22054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:36.198233 22054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:57:36.198335 22054 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 00:57:36.198383 22054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:36.198588 22054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:36.198644 22054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:57:36.198709 22054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:57:36.208352 22054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:36923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:36433
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:36433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:36.209564 22054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:36.210857 22054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:36.224170 22060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:36.229228 22054 server_base.cc:1034] running on GCE node
W20260810 00:57:36.229599 22063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:36.230046 22061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:36.231267 22054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:36.233031 22054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:36.234308 22054 hybrid_clock.cc:648] HybridClock initialized: now 1786323456234219 us; error 49 us; skew 500 ppm
I20260810 00:57:36.234717 22054 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:36 dist-test-slave-gp6n krb5kdc[22039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323456, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:36.241214 22054 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 00:57:36.242828 22054 webserver.cc:460] Webserver started at http://127.20.81.190:36815/ using document root <none> and password file <none>
I20260810 00:57:36.243571 22054 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:36.243687 22054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:36.244036 22054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:36.247197 22054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "d5357d3a65f943fe87bac39fb22f9b6a"
format_stamp: "Formatted at 2026-08-10 00:57:36 on dist-test-slave-gp6n"
server_key: "0c9794c386ceb805322c16fe8e47e3f7"
server_key_iv: "c88db88b463d31ee2445a4c81a26b2ad"
server_key_version: "encryptionkey@0"
I20260810 00:57:36.247992 22054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "d5357d3a65f943fe87bac39fb22f9b6a"
format_stamp: "Formatted at 2026-08-10 00:57:36 on dist-test-slave-gp6n"
server_key: "0c9794c386ceb805322c16fe8e47e3f7"
server_key_iv: "c88db88b463d31ee2445a4c81a26b2ad"
server_key_version: "encryptionkey@0"
I20260810 00:57:36.253273 22054 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 00:57:36.256896 22070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:36.258236 22054 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260810 00:57:36.258478 22054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "d5357d3a65f943fe87bac39fb22f9b6a"
format_stamp: "Formatted at 2026-08-10 00:57:36 on dist-test-slave-gp6n"
server_key: "0c9794c386ceb805322c16fe8e47e3f7"
server_key_iv: "c88db88b463d31ee2445a4c81a26b2ad"
server_key_version: "encryptionkey@0"
I20260810 00:57:36.258703 22054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:36.272205 22054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:36.276330 22054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:36.276619 22054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:36.298885 22054 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:36433
I20260810 00:57:36.298892 22138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:36433 every 8 connection(s)
I20260810 00:57:36.300313 22054 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:57:36.301689 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 22054
I20260810 00:57:36.301934 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 00:57:36.302804 20806 external_mini_cluster.cc:1428] Setting key 26bdbee9ace4922f18063cd4a46dc9dd
I20260810 00:57:36.305383 22139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 00:57:36 dist-test-slave-gp6n krb5kdc[22039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:36.319123 22139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a: Bootstrap starting.
I20260810 00:57:36.322594 22139 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:36.323670 22139 log.cc:824] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:36.326749 22143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:36.304956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36952 (local address 127.20.81.190:36433)
0810 00:57:36.306028 (+ 1072us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:36.306040 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:36.306080 (+ 40us) server_negotiation.cc:407] Connection header received
0810 00:57:36.307794 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:36.307814 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:36.308218 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:36.308553 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:36.310335 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:36.311788 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:36.313527 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:36.314197 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:36.314332 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:36.319342 (+ 5010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:36.319351 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:36.319366 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:36.319403 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:36.322473 (+ 3070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:36.323217 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:36.323226 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:36.323237 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:36.323419 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:36.323803 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:36.323811 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:36.323814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:36.324544 (+ 730us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:36.324875 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:36.325264 (+ 389us) server_negotiation.cc:299] Negotiation successful
0810 00:57:36.325657 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":843,"thread_start_us":653,"threads_started":1}
I20260810 00:57:36.327032 22139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a: No bootstrap required, opened a new log
I20260810 00:57:36.334331 22139 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5357d3a65f943fe87bac39fb22f9b6a" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 36433 } }
I20260810 00:57:36.334792 22139 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:36.334898 22139 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5357d3a65f943fe87bac39fb22f9b6a, State: Initialized, Role: FOLLOWER
I20260810 00:57:36.335510 22139 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5357d3a65f943fe87bac39fb22f9b6a" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 36433 } }
I20260810 00:57:36.335754 22139 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:57:36.335886 22139 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:57:36.336068 22139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:36.338165 22139 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5357d3a65f943fe87bac39fb22f9b6a" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 36433 } }
I20260810 00:57:36.338704 22139 leader_election.cc:302] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5357d3a65f943fe87bac39fb22f9b6a; no voters:
I20260810 00:57:36.339304 22139 leader_election.cc:288] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:57:36.339457 22145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:57:36.340207 22145 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a [term 1 LEADER]: Becoming Leader. State: Replica: d5357d3a65f943fe87bac39fb22f9b6a, State: Running, Role: LEADER
I20260810 00:57:36.340739 22145 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5357d3a65f943fe87bac39fb22f9b6a" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 36433 } }
I20260810 00:57:36.341449 22139 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:57:36.343935 22147 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5357d3a65f943fe87bac39fb22f9b6a. Latest consensus state: current_term: 1 leader_uuid: "d5357d3a65f943fe87bac39fb22f9b6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5357d3a65f943fe87bac39fb22f9b6a" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 36433 } } }
I20260810 00:57:36.343894 22146 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5357d3a65f943fe87bac39fb22f9b6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5357d3a65f943fe87bac39fb22f9b6a" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 36433 } } }
I20260810 00:57:36.344338 22147 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:36.344352 22146 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:36.346621 22154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:57:36.351176 22154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:57:36.359980 22154 catalog_manager.cc:1347] Generated new cluster ID: ecff0831eb4f416e8acd0b6ebb6079c2
I20260810 00:57:36.360102 22154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:57:36.382515 22154 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:57:36.383862 22154 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:57:36.392378 22154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a: Generated new TSK 0
I20260810 00:57:36.393229 22154 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.81.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
I20260810 00:57:36.453804 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:36433
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:36923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:37.001166 22168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:37.001639 22168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:37.001946 22168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:37.013953 22168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:37.014153 22168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:37.014384 22168 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 00:57:37.014597 22168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 00:57:37.031394 22168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:36923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.81.190:36433
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:37.033646 22168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:37.036115 22168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:37.059544 22173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:37.059598 22175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:37.059413 22178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:37.061299 22168 server_base.cc:1034] running on GCE node
I20260810 00:57:37.062302 22168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:37.063970 22168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:37.065446 22168 hybrid_clock.cc:648] HybridClock initialized: now 1786323457065305 us; error 82 us; skew 500 ppm
I20260810 00:57:37.065966 22168 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:37 dist-test-slave-gp6n krb5kdc[22039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323457, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:37.071534 22168 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 00:57:37.074115 22168 webserver.cc:460] Webserver started at http://127.20.81.129:35797/ using document root <none> and password file <none>
I20260810 00:57:37.075335 22168 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:37.075537 22168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:37.076191 22168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:37.081251 22168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "de9db0ea984343c69e5890549cbcff64"
format_stamp: "Formatted at 2026-08-10 00:57:37 on dist-test-slave-gp6n"
server_key: "4287ba9eb5eee62efa1e97e9844cf695"
server_key_iv: "12f2a10e52ff94bf1cb03308488f6006"
server_key_version: "encryptionkey@0"
I20260810 00:57:37.082710 22168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "de9db0ea984343c69e5890549cbcff64"
format_stamp: "Formatted at 2026-08-10 00:57:37 on dist-test-slave-gp6n"
server_key: "4287ba9eb5eee62efa1e97e9844cf695"
server_key_iv: "12f2a10e52ff94bf1cb03308488f6006"
server_key_version: "encryptionkey@0"
I20260810 00:57:37.092005 22168 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.007s sys 0.000s
I20260810 00:57:37.098148 22185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:37.100651 22168 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260810 00:57:37.101099 22168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "de9db0ea984343c69e5890549cbcff64"
format_stamp: "Formatted at 2026-08-10 00:57:37 on dist-test-slave-gp6n"
server_key: "4287ba9eb5eee62efa1e97e9844cf695"
server_key_iv: "12f2a10e52ff94bf1cb03308488f6006"
server_key_version: "encryptionkey@0"
I20260810 00:57:37.101655 22168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:37.159610 22168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:37.166868 22168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:37.167397 22168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:37.172410 22168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:57:37.172600 22168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:37.172869 22168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:57:37.173017 22168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 00:57:37 dist-test-slave-gp6n krb5kdc[22039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323457, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:37.213523 22195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:37.172818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:39207 (local address 127.20.81.190:36433)
0810 00:57:37.173703 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:37.173717 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:37.175987 (+ 2270us) server_negotiation.cc:407] Connection header received
0810 00:57:37.182041 (+ 6054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:37.182062 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:37.182209 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:37.182450 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:37.186599 (+ 4149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:37.187756 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:37.190774 (+ 3018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:37.191796 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:37.192047 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:37.201394 (+ 9347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:37.201404 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:37.201410 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:37.201539 (+ 129us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:37.207480 (+ 5941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:37.208769 (+ 1289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:37.208777 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:37.208794 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:37.209063 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:37.209911 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:37.209918 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:37.209951 (+ 33us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:37.210615 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:37.210907 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:37.212470 (+ 1563us) server_negotiation.cc:299] Negotiation successful
0810 00:57:37.212874 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":398,"threads_started":1}
I20260810 00:57:37.215281 22197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:37.174217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:36433 (local address 127.20.81.129:39207)
0810 00:57:37.175655 (+ 1438us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:37.175750 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:37.181306 (+ 5556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:37.183154 (+ 1848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:37.183173 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:37.184588 (+ 1415us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:37.186230 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:37.186263 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:37.188042 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:37.188065 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:37.190513 (+ 2448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:37.190544 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:37.192227 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:37.192237 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:37.192855 (+ 618us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:37.194337 (+ 1482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:37.194439 (+ 102us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:37.201006 (+ 6567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:37.207683 (+ 6677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:37.207711 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:37.207733 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:37.208572 (+ 839us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:37.209232 (+ 660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:37.209243 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:37.209247 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:37.209692 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:37.210869 (+ 1177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:37.210889 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:37.211806 (+ 917us) client_negotiation.cc:769] Sending connection context
0810 00:57:37.212301 (+ 495us) client_negotiation.cc:240] Negotiation successful
0810 00:57:37.213249 (+ 948us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":924,"thread_start_us":506,"threads_started":1}
I20260810 00:57:37.252938 22195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:37.225688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:53907 (local address 127.20.81.190:36433)
0810 00:57:37.225938 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:37.225944 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:37.228165 (+ 2221us) server_negotiation.cc:407] Connection header received
0810 00:57:37.228593 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:37.228602 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:37.228751 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:37.228905 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:37.231698 (+ 2793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:37.233189 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:37.237962 (+ 4773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:37.238764 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:37.239090 (+ 326us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:37.242343 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:37.242349 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:37.242353 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:37.242520 (+ 167us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:37.247940 (+ 5420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:37.249065 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:37.249082 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:37.249090 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:37.249443 (+ 353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:37.250434 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:37.250443 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:37.250446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:37.250898 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:37.251172 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:37.251852 (+ 680us) server_negotiation.cc:299] Negotiation successful
0810 00:57:37.252263 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 00:57:37.253116 22262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:37.226378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:36433 (local address 127.20.81.129:53907)
0810 00:57:37.228007 (+ 1629us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:37.228033 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:37.228283 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:37.229229 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:37.229232 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:37.230367 (+ 1135us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:37.231450 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:37.231480 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:37.233433 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:37.233453 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:37.237637 (+ 4184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:37.237672 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:37.239212 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:37.239231 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:37.239535 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:37.240629 (+ 1094us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:37.240660 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:37.241914 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:37.248180 (+ 6266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:37.248196 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:37.248202 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:37.248841 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:37.249671 (+ 830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:37.249691 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:37.249698 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:37.250114 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:37.251088 (+ 974us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:37.251093 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:37.251390 (+ 297us) client_negotiation.cc:769] Sending connection context
0810 00:57:37.251644 (+ 254us) client_negotiation.cc:240] Negotiation successful
0810 00:57:37.252253 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1358,"thread_start_us":957,"threads_started":1}
I20260810 00:57:37.269160 22168 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:39081
I20260810 00:57:37.269228 22319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:39081 every 8 connection(s)
I20260810 00:57:37.271134 22168 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 00:57:37.274474 22322 heartbeater.cc:347] Connected to a master server at 127.20.81.190:36433
I20260810 00:57:37.274773 22322 heartbeater.cc:464] Registering TS with master...
I20260810 00:57:37.275744 22322 heartbeater.cc:511] Master 127.20.81.190:36433 requested a full tablet report, sending...
I20260810 00:57:37.278339 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 22168
I20260810 00:57:37.278741 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 00:57:37.279245 22089 ts_manager.cc:194] Registered new tserver with Master: de9db0ea984343c69e5890549cbcff64 (127.20.81.129:39081)
I20260810 00:57:37.279979 20806 external_mini_cluster.cc:1428] Setting key 68ad90b49fc4cc04d034bdc3ae66dcbf
I20260810 00:57:37.282519 22089 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:39207
WARNING: no policy specified for kudu/127.20.81.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
I20260810 00:57:37.351109 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:36433
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:36923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:37.794227 22326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:37.794624 22326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:37.794808 22326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:37.801898 22326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:37.802052 22326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:37.802219 22326 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 00:57:37.802367 22326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 00:57:37.812392 22326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:36923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.81.190:36433
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:37.813715 22326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:37.815086 22326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:37.827746 22335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:37.827819 22332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:37.828087 22333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:37.829330 22326 server_base.cc:1034] running on GCE node
I20260810 00:57:37.830029 22326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:37.831082 22326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:37.832363 22326 hybrid_clock.cc:648] HybridClock initialized: now 1786323457832290 us; error 53 us; skew 500 ppm
I20260810 00:57:37.832703 22326 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:37 dist-test-slave-gp6n krb5kdc[22039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323457, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:37.836290 22326 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 00:57:37.837841 22326 webserver.cc:460] Webserver started at http://127.20.81.130:36269/ using document root <none> and password file <none>
I20260810 00:57:37.838665 22326 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:37.838811 22326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:37.839171 22326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:37.842505 22326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "21bed4c85c6249a5bbe628d811ae5e6c"
format_stamp: "Formatted at 2026-08-10 00:57:37 on dist-test-slave-gp6n"
server_key: "3cac4698b03d135984de28e0b7d8613b"
server_key_iv: "1088e3a187b6c4c834c7a7df71fb0aec"
server_key_version: "encryptionkey@0"
I20260810 00:57:37.843420 22326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "21bed4c85c6249a5bbe628d811ae5e6c"
format_stamp: "Formatted at 2026-08-10 00:57:37 on dist-test-slave-gp6n"
server_key: "3cac4698b03d135984de28e0b7d8613b"
server_key_iv: "1088e3a187b6c4c834c7a7df71fb0aec"
server_key_version: "encryptionkey@0"
I20260810 00:57:37.849107 22326 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 00:57:37.852905 22344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:37.854382 22326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 00:57:37.854740 22326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "21bed4c85c6249a5bbe628d811ae5e6c"
format_stamp: "Formatted at 2026-08-10 00:57:37 on dist-test-slave-gp6n"
server_key: "3cac4698b03d135984de28e0b7d8613b"
server_key_iv: "1088e3a187b6c4c834c7a7df71fb0aec"
server_key_version: "encryptionkey@0"
I20260810 00:57:37.854990 22326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:37.882973 22326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:37.887583 22326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:37.887885 22326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:37.890648 22326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:57:37.890758 22326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:37.890952 22326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:57:37.891029 22326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 00:57:37 dist-test-slave-gp6n krb5kdc[22039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323457, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:37.918102 22356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:37.892306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:36433 (local address 127.20.81.130:41689)
0810 00:57:37.893158 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:37.893194 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:37.895132 (+ 1938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:37.895948 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:37.895963 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:37.896744 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:37.897643 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:37.897663 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:37.899067 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:37.899075 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:37.901862 (+ 2787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:37.901879 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:37.903229 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:37.903241 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:37.903626 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:37.904501 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:37.904524 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:37.908215 (+ 3691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:37.912236 (+ 4021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:37.912249 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:37.912271 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:37.913049 (+ 778us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:37.913467 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:37.913475 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:37.913481 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:37.913828 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:37.914375 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:37.914387 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:37.915014 (+ 627us) client_negotiation.cc:769] Sending connection context
0810 00:57:37.915414 (+ 400us) client_negotiation.cc:240] Negotiation successful
0810 00:57:37.916122 (+ 708us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":366,"threads_started":1}
I20260810 00:57:37.919647 22354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:37.891784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:41689 (local address 127.20.81.190:36433)
0810 00:57:37.892273 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:37.892278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:37.893316 (+ 1038us) server_negotiation.cc:407] Connection header received
0810 00:57:37.895412 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:37.895418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:37.895512 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:37.895641 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:37.897824 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:37.898885 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:37.902032 (+ 3147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:37.902778 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:37.902915 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:37.908465 (+ 5550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:37.908475 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:37.908483 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:37.908556 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:37.912094 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:37.913184 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:37.913188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:37.913191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:37.913314 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:37.913968 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:37.913972 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:37.913974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:37.914237 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:37.914353 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:37.918801 (+ 4448us) server_negotiation.cc:299] Negotiation successful
0810 00:57:37.919142 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":281,"threads_started":1}
I20260810 00:57:37.942224 22354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:37.925766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:35869 (local address 127.20.81.190:36433)
0810 00:57:37.926007 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:37.926014 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:37.927299 (+ 1285us) server_negotiation.cc:407] Connection header received
0810 00:57:37.927485 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:37.927489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:37.927554 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:37.927639 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:37.929224 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:37.930022 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:37.932301 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:37.932818 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:37.932979 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:37.935842 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:37.935849 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:37.935870 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:37.935917 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:37.939494 (+ 3577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:37.940167 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:37.940172 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:37.940176 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:37.940311 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:37.940688 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:37.940693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:37.940696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:37.941028 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:37.941184 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:37.941605 (+ 421us) server_negotiation.cc:299] Negotiation successful
0810 00:57:37.941837 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 00:57:37.942315 22448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:37.926187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:36433 (local address 127.20.81.130:35869)
0810 00:57:37.927162 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:37.927193 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:37.927368 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:37.927705 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:37.927713 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:37.928491 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:37.929013 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:37.929032 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:37.930179 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:37.930187 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:37.932145 (+ 1958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:37.932166 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:37.933069 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:37.933079 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:37.933342 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:37.934099 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:37.934130 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:37.935612 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:37.939636 (+ 4024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:37.939641 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:37.939646 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:37.940035 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:37.940427 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:37.940433 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:37.940436 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:37.940574 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:37.941148 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:37.941153 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:37.941334 (+ 181us) client_negotiation.cc:769] Sending connection context
0810 00:57:37.941480 (+ 146us) client_negotiation.cc:240] Negotiation successful
0810 00:57:37.941849 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"thread_start_us":601,"threads_started":1}
I20260810 00:57:37.948179 22326 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:40029
I20260810 00:57:37.948259 22475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:40029 every 8 connection(s)
I20260810 00:57:37.949429 22326 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 00:57:37.951581 22477 heartbeater.cc:347] Connected to a master server at 127.20.81.190:36433
I20260810 00:57:37.951841 22477 heartbeater.cc:464] Registering TS with master...
I20260810 00:57:37.952533 22477 heartbeater.cc:511] Master 127.20.81.190:36433 requested a full tablet report, sending...
I20260810 00:57:37.953339 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 22326
I20260810 00:57:37.953560 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 00:57:37.954043 20806 external_mini_cluster.cc:1428] Setting key 16866cb29a173973aef402ca9df24b11
I20260810 00:57:37.954106 22089 ts_manager.cc:194] Registered new tserver with Master: 21bed4c85c6249a5bbe628d811ae5e6c (127.20.81.130:40029)
I20260810 00:57:37.955005 22089 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:41689
WARNING: no policy specified for kudu/127.20.81.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
I20260810 00:57:38.001333 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:36433
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:36923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:57:38.287290 22322 heartbeater.cc:503] Master 127.20.81.190:36433 was elected leader, sending a full tablet report...
W20260810 00:57:38.346202 22482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:38.346573 22482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:38.346769 22482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:38.353929 22482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:38.354038 22482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:38.354122 22482 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 00:57:38.354203 22482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 00:57:38.364455 22482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:36923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.81.190:36433
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:38.365756 22482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:38.367131 22482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:38.380347 22487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:38.380314 22490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:38.380275 22488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:38.381531 22482 server_base.cc:1034] running on GCE node
I20260810 00:57:38.382267 22482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:38.383323 22482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:38.384603 22482 hybrid_clock.cc:648] HybridClock initialized: now 1786323458384536 us; error 36 us; skew 500 ppm
I20260810 00:57:38.384948 22482 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:38 dist-test-slave-gp6n krb5kdc[22039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323458, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:38.389485 22482 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 00:57:38.391052 22482 webserver.cc:460] Webserver started at http://127.20.81.131:43917/ using document root <none> and password file <none>
I20260810 00:57:38.391727 22482 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:38.391824 22482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:38.392089 22482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:38.395144 22482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "960a71f15b63450a8b0f706715a2f152"
format_stamp: "Formatted at 2026-08-10 00:57:38 on dist-test-slave-gp6n"
server_key: "3f905f62cd7e5dd6a3382e40793ee503"
server_key_iv: "c9a8ace9fc80001139dea07ba6356c74"
server_key_version: "encryptionkey@0"
I20260810 00:57:38.395962 22482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "960a71f15b63450a8b0f706715a2f152"
format_stamp: "Formatted at 2026-08-10 00:57:38 on dist-test-slave-gp6n"
server_key: "3f905f62cd7e5dd6a3382e40793ee503"
server_key_iv: "c9a8ace9fc80001139dea07ba6356c74"
server_key_version: "encryptionkey@0"
I20260810 00:57:38.401448 22482 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 00:57:38.405011 22498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:38.406563 22482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:57:38.406744 22482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "960a71f15b63450a8b0f706715a2f152"
format_stamp: "Formatted at 2026-08-10 00:57:38 on dist-test-slave-gp6n"
server_key: "3f905f62cd7e5dd6a3382e40793ee503"
server_key_iv: "c9a8ace9fc80001139dea07ba6356c74"
server_key_version: "encryptionkey@0"
I20260810 00:57:38.406970 22482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:38.419746 22482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:38.424371 22482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:38.424651 22482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:38.427412 22482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:57:38.427492 22482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:38.427716 22482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:57:38.427795 22482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 00:57:38 dist-test-slave-gp6n krb5kdc[22039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323458, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:38.451787 22354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:38.427804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:40341 (local address 127.20.81.190:36433)
0810 00:57:38.428011 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:38.428020 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:38.429603 (+ 1583us) server_negotiation.cc:407] Connection header received
0810 00:57:38.431685 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:38.431695 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:38.431797 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:38.431913 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:38.433789 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.435151 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.437920 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.438683 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.438849 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:38.443896 (+ 5047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:38.443903 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:38.443907 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:38.443947 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:38.448224 (+ 4277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:38.448928 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:38.448936 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:38.448943 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:38.449149 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:38.449576 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:38.449584 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:38.449589 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:38.450042 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:38.450213 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:38.451037 (+ 824us) server_negotiation.cc:299] Negotiation successful
0810 00:57:38.451293 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 00:57:38.453330 22508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:38.428470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:36433 (local address 127.20.81.131:40341)
0810 00:57:38.429388 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:38.429458 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:38.431366 (+ 1908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:38.432167 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:38.432177 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:38.432981 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:38.433598 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:38.433614 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.435418 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.435425 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:38.437739 (+ 2314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:38.437766 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.438967 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.438978 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:38.439309 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:38.440111 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:38.440143 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:38.443676 (+ 3533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:38.448386 (+ 4710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:38.448393 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:38.448404 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:38.448811 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:38.449268 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:38.449281 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:38.449284 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:38.449468 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:38.450192 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:38.450197 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:38.450614 (+ 417us) client_negotiation.cc:769] Sending connection context
0810 00:57:38.450925 (+ 311us) client_negotiation.cc:240] Negotiation successful
0810 00:57:38.451552 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":415,"threads_started":1}
I20260810 00:57:38.474378 22482 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:43717
I20260810 00:57:38.474443 22639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:43717 every 8 connection(s)
I20260810 00:57:38.475674 22482 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 00:57:38.476637 22354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:38.460704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:56201 (local address 127.20.81.190:36433)
0810 00:57:38.460882 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:38.460888 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:38.461837 (+ 949us) server_negotiation.cc:407] Connection header received
0810 00:57:38.462135 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:38.462141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:38.462212 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:38.462264 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:38.463928 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.465202 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.466982 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.467602 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.467735 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:38.470545 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:38.470550 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:38.470553 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:38.470582 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:38.473700 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:38.474458 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:38.474465 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:38.474468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:38.474646 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:38.475031 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:38.475037 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:38.475041 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:38.475360 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:38.475526 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:38.475983 (+ 457us) server_negotiation.cc:299] Negotiation successful
0810 00:57:38.476216 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 00:57:38.476655 22599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:38.460795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:36433 (local address 127.20.81.131:56201)
0810 00:57:38.461697 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:38.461720 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:38.461955 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:38.462555 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:38.462562 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:38.463333 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:38.463740 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:38.463749 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.465346 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.465350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:38.466753 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:38.466765 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.467937 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.467943 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:38.468207 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:38.468937 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:38.468968 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:38.470239 (+ 1271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:38.473830 (+ 3591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:38.473835 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:38.473839 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:38.474287 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:38.474761 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:38.474766 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:38.474770 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:38.474899 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:38.475499 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:38.475504 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:38.475694 (+ 190us) client_negotiation.cc:769] Sending connection context
0810 00:57:38.475849 (+ 155us) client_negotiation.cc:240] Negotiation successful
0810 00:57:38.476128 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":584,"threads_started":1}
I20260810 00:57:38.478220 22640 heartbeater.cc:347] Connected to a master server at 127.20.81.190:36433
I20260810 00:57:38.478521 22640 heartbeater.cc:464] Registering TS with master...
I20260810 00:57:38.479301 22640 heartbeater.cc:511] Master 127.20.81.190:36433 requested a full tablet report, sending...
I20260810 00:57:38.480698 22088 ts_manager.cc:194] Registered new tserver with Master: 960a71f15b63450a8b0f706715a2f152 (127.20.81.131:43717)
I20260810 00:57:38.481588 22088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:40341
I20260810 00:57:38.485790 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 22482
I20260810 00:57:38.486013 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 00:57:38.486526 20806 external_mini_cluster.cc:1428] Setting key 15ba7548e75477fc8912046a5314cf29
I20260810 00:57:38.489463 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 00:57:38 dist-test-slave-gp6n krb5kdc[22039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323458, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 00:57:38 dist-test-slave-gp6n krb5kdc[22039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323458, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:38.539914 22354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:38.525563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36962 (local address 127.20.81.190:36433)
0810 00:57:38.525819 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:38.525826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:38.525998 (+ 172us) server_negotiation.cc:407] Connection header received
0810 00:57:38.526230 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:38.526234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:38.526302 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:38.526412 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:38.527907 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.528661 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.529952 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.530359 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.530552 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:38.534360 (+ 3808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:38.534367 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:38.534373 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:38.534472 (+ 99us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:38.536980 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:38.537610 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:38.537617 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:38.537621 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:38.537848 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:38.538240 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:38.538246 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:38.538249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:38.538657 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:38.538996 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:38.539257 (+ 261us) server_negotiation.cc:299] Negotiation successful
0810 00:57:38.539500 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 00:57:38.544356 22088 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:57:38.552300 22088 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:57:38.584323 22655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:38.565485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:43717 (local address 127.0.0.1:38136)
0810 00:57:38.572837 (+ 7352us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:38.572883 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:38.573092 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:38.574382 (+ 1290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:38.574393 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:38.574468 (+ 75us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:57:38.574927 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:38.574944 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.577408 (+ 2464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.577428 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:38.581139 (+ 3711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:38.581155 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.583366 (+ 2211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.583374 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:38.583580 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:38.583631 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 00:57:38.583760 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 00:57:38.583943 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7065,"thread_start_us":363,"threads_started":1}
I20260810 00:57:38.584460 22656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:38.565667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38136 (local address 127.20.81.131:43717)
0810 00:57:38.572759 (+ 7092us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:38.572775 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:38.573297 (+ 522us) server_negotiation.cc:407] Connection header received
0810 00:57:38.573432 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:38.573443 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:38.573740 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:38.573937 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:57:38.575208 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.577217 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.581309 (+ 4092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.582835 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.582963 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:38.583243 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:38.583863 (+ 620us) server_negotiation.cc:299] Negotiation successful
0810 00:57:38.584044 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6811,"thread_start_us":6522,"threads_started":1}
I20260810 00:57:38.589721 22657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:38.566350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:40029 (local address 127.0.0.1:54480)
0810 00:57:38.580712 (+ 14362us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:38.580736 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:38.580912 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:38.582100 (+ 1188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:38.582105 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:38.582130 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:57:38.582693 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:38.582711 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.584323 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.584328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:38.586965 (+ 2637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:38.586978 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.588922 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.588929 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:38.589085 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:38.589117 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 00:57:38.589211 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 00:57:38.589353 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14089,"mutex_wait_us":19,"spinlock_wait_cycles":4480,"thread_start_us":13667,"threads_started":1}
I20260810 00:57:38.589150 22556 tablet_service.cc:1467] Processing CreateTablet for tablet 0a0a352416df4edc9c58c59d4dd4e103 (DEFAULT_TABLE table=test-table [id=8a85b0ace4894b6b876330aaba936c6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:57:38.589876 22658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:38.566601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54480 (local address 127.20.81.130:40029)
0810 00:57:38.578900 (+ 12299us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:38.578928 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:38.581085 (+ 2157us) server_negotiation.cc:407] Connection header received
0810 00:57:38.581660 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:38.581669 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:38.581906 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:38.582100 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:57:38.582925 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.584137 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.587128 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.588702 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.588871 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:38.589079 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:38.589335 (+ 256us) server_negotiation.cc:299] Negotiation successful
0810 00:57:38.589540 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12064,"thread_start_us":12465,"threads_started":1}
I20260810 00:57:38.590762 22661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:38.567566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39118 (local address 127.20.81.129:39081)
0810 00:57:38.579649 (+ 12083us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:38.579661 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:38.581366 (+ 1705us) server_negotiation.cc:407] Connection header received
0810 00:57:38.581488 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:38.581496 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:38.581768 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:38.581984 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:57:38.583051 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.585236 (+ 2185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.587376 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.589359 (+ 1983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.589503 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:38.589705 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:38.590300 (+ 595us) server_negotiation.cc:299] Negotiation successful
0810 00:57:38.590494 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11903,"thread_start_us":11759,"threads_started":1}
I20260810 00:57:38.590770 22660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:38.566949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:39081 (local address 127.0.0.1:39118)
0810 00:57:38.581062 (+ 14113us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:38.581080 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:38.581206 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:38.582242 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:38.582247 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:38.582291 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:57:38.582874 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:38.582888 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.585374 (+ 2486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.585380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:38.587196 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:38.587212 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.589867 (+ 2655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.589875 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:38.590092 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:38.590132 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 00:57:38.590234 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 00:57:38.590379 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13949,"mutex_wait_us":18,"thread_start_us":13795,"threads_started":1}
I20260810 00:57:38.590822 22556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a0a352416df4edc9c58c59d4dd4e103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:38.593863 22402 tablet_service.cc:1467] Processing CreateTablet for tablet 0a0a352416df4edc9c58c59d4dd4e103 (DEFAULT_TABLE table=test-table [id=8a85b0ace4894b6b876330aaba936c6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:57:38.594623 22243 tablet_service.cc:1467] Processing CreateTablet for tablet 0a0a352416df4edc9c58c59d4dd4e103 (DEFAULT_TABLE table=test-table [id=8a85b0ace4894b6b876330aaba936c6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:57:38.595459 22402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a0a352416df4edc9c58c59d4dd4e103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:38.596043 22243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a0a352416df4edc9c58c59d4dd4e103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:38.602362 22662 tablet_bootstrap.cc:492] T 0a0a352416df4edc9c58c59d4dd4e103 P 960a71f15b63450a8b0f706715a2f152: Bootstrap starting.
I20260810 00:57:38.605921 22662 tablet_bootstrap.cc:654] T 0a0a352416df4edc9c58c59d4dd4e103 P 960a71f15b63450a8b0f706715a2f152: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:38.607009 22662 log.cc:824] T 0a0a352416df4edc9c58c59d4dd4e103 P 960a71f15b63450a8b0f706715a2f152: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:38.614378 22664 tablet_bootstrap.cc:492] T 0a0a352416df4edc9c58c59d4dd4e103 P de9db0ea984343c69e5890549cbcff64: Bootstrap starting.
I20260810 00:57:38.615617 22665 tablet_bootstrap.cc:492] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c: Bootstrap starting.
I20260810 00:57:38.617445 22664 tablet_bootstrap.cc:654] T 0a0a352416df4edc9c58c59d4dd4e103 P de9db0ea984343c69e5890549cbcff64: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:38.618443 22665 tablet_bootstrap.cc:654] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:38.618814 22664 log.cc:824] T 0a0a352416df4edc9c58c59d4dd4e103 P de9db0ea984343c69e5890549cbcff64: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:38.620111 22665 log.cc:824] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:38.620398 22662 tablet_bootstrap.cc:492] T 0a0a352416df4edc9c58c59d4dd4e103 P 960a71f15b63450a8b0f706715a2f152: No bootstrap required, opened a new log
I20260810 00:57:38.620728 22662 ts_tablet_manager.cc:1397] T 0a0a352416df4edc9c58c59d4dd4e103 P 960a71f15b63450a8b0f706715a2f152: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.004s
I20260810 00:57:38.622103 22664 tablet_bootstrap.cc:492] T 0a0a352416df4edc9c58c59d4dd4e103 P de9db0ea984343c69e5890549cbcff64: No bootstrap required, opened a new log
I20260810 00:57:38.622390 22664 ts_tablet_manager.cc:1397] T 0a0a352416df4edc9c58c59d4dd4e103 P de9db0ea984343c69e5890549cbcff64: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260810 00:57:38.623515 22665 tablet_bootstrap.cc:492] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c: No bootstrap required, opened a new log
I20260810 00:57:38.623889 22665 ts_tablet_manager.cc:1397] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.004s
I20260810 00:57:38.631870 22665 raft_consensus.cc:348] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } } peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } }
I20260810 00:57:38.632315 22665 raft_consensus.cc:374] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:38.632463 22665 raft_consensus.cc:729] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21bed4c85c6249a5bbe628d811ae5e6c, State: Initialized, Role: FOLLOWER
I20260810 00:57:38.632860 22662 raft_consensus.cc:348] T 0a0a352416df4edc9c58c59d4dd4e103 P 960a71f15b63450a8b0f706715a2f152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } } peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } }
I20260810 00:57:38.633296 22662 raft_consensus.cc:374] T 0a0a352416df4edc9c58c59d4dd4e103 P 960a71f15b63450a8b0f706715a2f152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:38.633162 22665 consensus_queue.cc:260] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } } peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } }
I20260810 00:57:38.633437 22662 raft_consensus.cc:729] T 0a0a352416df4edc9c58c59d4dd4e103 P 960a71f15b63450a8b0f706715a2f152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 960a71f15b63450a8b0f706715a2f152, State: Initialized, Role: FOLLOWER
I20260810 00:57:38.634150 22662 consensus_queue.cc:260] T 0a0a352416df4edc9c58c59d4dd4e103 P 960a71f15b63450a8b0f706715a2f152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } } peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } }
I20260810 00:57:38.634187 22664 raft_consensus.cc:348] T 0a0a352416df4edc9c58c59d4dd4e103 P de9db0ea984343c69e5890549cbcff64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } } peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } }
I20260810 00:57:38.634665 22664 raft_consensus.cc:374] T 0a0a352416df4edc9c58c59d4dd4e103 P de9db0ea984343c69e5890549cbcff64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:38.634840 22664 raft_consensus.cc:729] T 0a0a352416df4edc9c58c59d4dd4e103 P de9db0ea984343c69e5890549cbcff64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de9db0ea984343c69e5890549cbcff64, State: Initialized, Role: FOLLOWER
I20260810 00:57:38.635933 22477 heartbeater.cc:503] Master 127.20.81.190:36433 was elected leader, sending a full tablet report...
I20260810 00:57:38.635752 22664 consensus_queue.cc:260] T 0a0a352416df4edc9c58c59d4dd4e103 P de9db0ea984343c69e5890549cbcff64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } } peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } }
I20260810 00:57:38.636761 22665 ts_tablet_manager.cc:1428] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c: Time spent starting tablet: real 0.013s user 0.005s sys 0.007s
I20260810 00:57:38.639078 22664 ts_tablet_manager.cc:1428] T 0a0a352416df4edc9c58c59d4dd4e103 P de9db0ea984343c69e5890549cbcff64: Time spent starting tablet: real 0.017s user 0.014s sys 0.002s
I20260810 00:57:38.639981 22640 heartbeater.cc:503] Master 127.20.81.190:36433 was elected leader, sending a full tablet report...
I20260810 00:57:38.640583 22662 ts_tablet_manager.cc:1428] T 0a0a352416df4edc9c58c59d4dd4e103 P 960a71f15b63450a8b0f706715a2f152: Time spent starting tablet: real 0.020s user 0.008s sys 0.010s
W20260810 00:57:38.701505 22478 tablet.cc:2369] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:57:38.728463 22641 tablet.cc:2369] T 0a0a352416df4edc9c58c59d4dd4e103 P 960a71f15b63450a8b0f706715a2f152: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:57:38.774999 22323 tablet.cc:2369] T 0a0a352416df4edc9c58c59d4dd4e103 P de9db0ea984343c69e5890549cbcff64: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:57:38.798913 22668 raft_consensus.cc:482] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:57:38.799233 22668 raft_consensus.cc:504] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } } peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } }
I20260810 00:57:38.801057 22668 leader_election.cc:288] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 960a71f15b63450a8b0f706715a2f152 (127.20.81.131:43717), de9db0ea984343c69e5890549cbcff64 (127.20.81.129:39081)
I20260810 00:57:38.808372 22656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:38.801482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:39163 (local address 127.20.81.131:43717)
0810 00:57:38.801747 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:38.801757 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:38.802285 (+ 528us) server_negotiation.cc:407] Connection header received
0810 00:57:38.802547 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:38.802553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:38.802634 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:38.802709 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:57:38.803460 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.804433 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.806453 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.807389 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.807514 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:38.807567 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:38.807981 (+ 414us) server_negotiation.cc:299] Negotiation successful
0810 00:57:38.808116 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 00:57:38.808506 22674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:38.801429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:43717 (local address 127.20.81.130:39163)
0810 00:57:38.802114 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:38.802158 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:38.802330 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:38.802890 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:38.802893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:38.802916 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:57:38.803300 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:38.803313 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.804568 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.804573 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:38.806260 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:38.806272 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.807617 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.807624 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:38.807809 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:38.807843 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 00:57:38.807946 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 00:57:38.808089 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"mutex_wait_us":21,"thread_start_us":385,"threads_started":1}
I20260810 00:57:38.809020 22675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:38.802196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:39081 (local address 127.20.81.130:52149)
0810 00:57:38.802864 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:38.802878 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:38.803001 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:38.803408 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:38.803413 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:38.803439 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:57:38.803876 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:38.803888 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.805064 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.805068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:38.806744 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:38.806760 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.808065 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.808072 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:38.808235 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:38.808323 (+ 88us) client_negotiation.cc:769] Sending connection context
0810 00:57:38.808436 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 00:57:38.808589 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"mutex_wait_us":2,"thread_start_us":353,"threads_started":1}
I20260810 00:57:38.809209 22661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:38.802390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:52149 (local address 127.20.81.129:39081)
0810 00:57:38.802638 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:38.802644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:38.802995 (+ 351us) server_negotiation.cc:407] Connection header received
0810 00:57:38.803118 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:38.803123 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:38.803216 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:38.803316 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:57:38.804037 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.804919 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.806890 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.807823 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.807989 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:38.808055 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:38.808487 (+ 432us) server_negotiation.cc:299] Negotiation successful
0810 00:57:38.808985 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 00:57:38.809769 22586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a0a352416df4edc9c58c59d4dd4e103" candidate_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "960a71f15b63450a8b0f706715a2f152" is_pre_election: true
I20260810 00:57:38.810258 22272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a0a352416df4edc9c58c59d4dd4e103" candidate_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de9db0ea984343c69e5890549cbcff64" is_pre_election: true
I20260810 00:57:38.810467 22586 raft_consensus.cc:2454] T 0a0a352416df4edc9c58c59d4dd4e103 P 960a71f15b63450a8b0f706715a2f152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21bed4c85c6249a5bbe628d811ae5e6c in term 0.
I20260810 00:57:38.810787 22272 raft_consensus.cc:2454] T 0a0a352416df4edc9c58c59d4dd4e103 P de9db0ea984343c69e5890549cbcff64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21bed4c85c6249a5bbe628d811ae5e6c in term 0.
I20260810 00:57:38.811173 22348 leader_election.cc:302] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21bed4c85c6249a5bbe628d811ae5e6c, 960a71f15b63450a8b0f706715a2f152; no voters:
I20260810 00:57:38.811650 22668 raft_consensus.cc:2781] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:57:38.811841 22668 raft_consensus.cc:482] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:57:38.811975 22668 raft_consensus.cc:3037] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:38.814216 22668 raft_consensus.cc:504] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } } peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } }
I20260810 00:57:38.814828 22668 leader_election.cc:288] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c [CANDIDATE]: Term 1 election: Requested vote from peers 960a71f15b63450a8b0f706715a2f152 (127.20.81.131:43717), de9db0ea984343c69e5890549cbcff64 (127.20.81.129:39081)
I20260810 00:57:38.815274 22586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a0a352416df4edc9c58c59d4dd4e103" candidate_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "960a71f15b63450a8b0f706715a2f152"
I20260810 00:57:38.815424 22272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a0a352416df4edc9c58c59d4dd4e103" candidate_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de9db0ea984343c69e5890549cbcff64"
I20260810 00:57:38.815485 22586 raft_consensus.cc:3037] T 0a0a352416df4edc9c58c59d4dd4e103 P 960a71f15b63450a8b0f706715a2f152 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:38.815658 22272 raft_consensus.cc:3037] T 0a0a352416df4edc9c58c59d4dd4e103 P de9db0ea984343c69e5890549cbcff64 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:38.817709 22586 raft_consensus.cc:2454] T 0a0a352416df4edc9c58c59d4dd4e103 P 960a71f15b63450a8b0f706715a2f152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21bed4c85c6249a5bbe628d811ae5e6c in term 1.
I20260810 00:57:38.817808 22272 raft_consensus.cc:2454] T 0a0a352416df4edc9c58c59d4dd4e103 P de9db0ea984343c69e5890549cbcff64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21bed4c85c6249a5bbe628d811ae5e6c in term 1.
I20260810 00:57:38.818120 22348 leader_election.cc:302] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21bed4c85c6249a5bbe628d811ae5e6c, 960a71f15b63450a8b0f706715a2f152; no voters:
I20260810 00:57:38.818547 22668 raft_consensus.cc:2781] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:57:38.818981 22668 raft_consensus.cc:686] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c [term 1 LEADER]: Becoming Leader. State: Replica: 21bed4c85c6249a5bbe628d811ae5e6c, State: Running, Role: LEADER
I20260810 00:57:38.819465 22668 consensus_queue.cc:237] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } } peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } }
I20260810 00:57:38.824254 22087 catalog_manager.cc:5495] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 21bed4c85c6249a5bbe628d811ae5e6c (127.20.81.130). New cstate: current_term: 1 leader_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } health_report { overall_health: HEALTHY } } }
I20260810 00:57:38.896950 22658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:38.891317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54496 (local address 127.20.81.130:40029)
0810 00:57:38.891587 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:38.891597 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:38.891629 (+ 32us) server_negotiation.cc:407] Connection header received
0810 00:57:38.891895 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:38.891900 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:38.891988 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:38.892106 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:38.892490 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.893245 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.894853 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:38.895254 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:38.895360 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:38.895892 (+ 532us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:38.896401 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:38.896513 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:38.896659 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 00:57:38.896729 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260810 00:57:38.905160 22272 raft_consensus.cc:1261] T 0a0a352416df4edc9c58c59d4dd4e103 P de9db0ea984343c69e5890549cbcff64 [term 1 FOLLOWER]: Refusing update from remote peer 21bed4c85c6249a5bbe628d811ae5e6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:57:38.905158 22586 raft_consensus.cc:1261] T 0a0a352416df4edc9c58c59d4dd4e103 P 960a71f15b63450a8b0f706715a2f152 [term 1 FOLLOWER]: Refusing update from remote peer 21bed4c85c6249a5bbe628d811ae5e6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:57:38.906056 22676 consensus_queue.cc:1035] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:57:38.906494 22668 consensus_queue.cc:1035] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:57:38.917716 22679 mvcc.cc:205] Tried to move back new op lower bound from 7316780887665975296 to 7316780887328645120. Current Snapshot: MvccSnapshot[applied={T|T < 7316780887665975296}]
I20260810 00:57:38.920869 22680 mvcc.cc:205] Tried to move back new op lower bound from 7316780887665975296 to 7316780887328645120. Current Snapshot: MvccSnapshot[applied={T|T < 7316780887665975296}]
I20260810 00:57:38.936316 22088 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:57:38.936597 22088 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:57:38.939986 22088 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a: Sending DeleteTablet for 3 replicas of tablet 0a0a352416df4edc9c58c59d4dd4e103
I20260810 00:57:38.941075 22556 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a0a352416df4edc9c58c59d4dd4e103 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:57:38 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:38136
I20260810 00:57:38.941076 22402 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a0a352416df4edc9c58c59d4dd4e103 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:57:38 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:54480
I20260810 00:57:38.941152 22243 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a0a352416df4edc9c58c59d4dd4e103 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:57:38 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:39118
I20260810 00:57:38.946702 22088 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:57:38.948305 22088 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:57:38.956588 22693 tablet_replica.cc:330] T 0a0a352416df4edc9c58c59d4dd4e103 P de9db0ea984343c69e5890549cbcff64: stopping tablet replica
I20260810 00:57:38.956880 22692 tablet_replica.cc:330] T 0a0a352416df4edc9c58c59d4dd4e103 P 960a71f15b63450a8b0f706715a2f152: stopping tablet replica
I20260810 00:57:38.957067 22693 raft_consensus.cc:2229] T 0a0a352416df4edc9c58c59d4dd4e103 P de9db0ea984343c69e5890549cbcff64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:57:38.957430 22692 raft_consensus.cc:2229] T 0a0a352416df4edc9c58c59d4dd4e103 P 960a71f15b63450a8b0f706715a2f152 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:57:38.957597 22693 raft_consensus.cc:2258] T 0a0a352416df4edc9c58c59d4dd4e103 P de9db0ea984343c69e5890549cbcff64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:57:38.957967 22692 raft_consensus.cc:2258] T 0a0a352416df4edc9c58c59d4dd4e103 P 960a71f15b63450a8b0f706715a2f152 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:57:38.958848 22243 tablet_service.cc:1467] Processing CreateTablet for tablet a5d5c5e4fcb74edeb73c81db5d9d77f5 (DEFAULT_TABLE table=test-table [id=223e825673114643b26c11c110b83512]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:57:38.959038 22690 tablet_replica.cc:330] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c: stopping tablet replica
I20260810 00:57:38.959246 22556 tablet_service.cc:1467] Processing CreateTablet for tablet a5d5c5e4fcb74edeb73c81db5d9d77f5 (DEFAULT_TABLE table=test-table [id=223e825673114643b26c11c110b83512]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:57:38.959369 22243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5d5c5e4fcb74edeb73c81db5d9d77f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:38.959411 22690 raft_consensus.cc:2229] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:57:38.959688 22556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5d5c5e4fcb74edeb73c81db5d9d77f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:38.960142 22690 raft_consensus.cc:2258] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:57:38.963552 22693 ts_tablet_manager.cc:1905] T 0a0a352416df4edc9c58c59d4dd4e103 P de9db0ea984343c69e5890549cbcff64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:57:38.966172 22662 tablet_bootstrap.cc:492] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152: Bootstrap starting.
I20260810 00:57:38.966750 22402 tablet_service.cc:1467] Processing CreateTablet for tablet a5d5c5e4fcb74edeb73c81db5d9d77f5 (DEFAULT_TABLE table=test-table [id=223e825673114643b26c11c110b83512]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:57:38.967206 22402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5d5c5e4fcb74edeb73c81db5d9d77f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:38.968024 22662 tablet_bootstrap.cc:654] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:38.968135 22692 ts_tablet_manager.cc:1905] T 0a0a352416df4edc9c58c59d4dd4e103 P 960a71f15b63450a8b0f706715a2f152: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:57:38.972792 22693 ts_tablet_manager.cc:1918] T 0a0a352416df4edc9c58c59d4dd4e103 P de9db0ea984343c69e5890549cbcff64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:57:38.973057 22693 log.cc:1196] T 0a0a352416df4edc9c58c59d4dd4e103 P de9db0ea984343c69e5890549cbcff64: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-0/wal/wals/0a0a352416df4edc9c58c59d4dd4e103
I20260810 00:57:38.973749 22665 tablet_bootstrap.cc:492] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 21bed4c85c6249a5bbe628d811ae5e6c: Bootstrap starting.
I20260810 00:57:38.973768 22693 ts_tablet_manager.cc:1939] T 0a0a352416df4edc9c58c59d4dd4e103 P de9db0ea984343c69e5890549cbcff64: Deleting consensus metadata
I20260810 00:57:38.975523 22665 tablet_bootstrap.cc:654] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 21bed4c85c6249a5bbe628d811ae5e6c: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:38.975746 22690 ts_tablet_manager.cc:1905] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:57:38.976048 22074 catalog_manager.cc:4841] TS de9db0ea984343c69e5890549cbcff64 (127.20.81.129:39081): tablet 0a0a352416df4edc9c58c59d4dd4e103 (table test-table [id=8a85b0ace4894b6b876330aaba936c6c]) successfully deleted
I20260810 00:57:38.976912 22664 tablet_bootstrap.cc:492] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P de9db0ea984343c69e5890549cbcff64: Bootstrap starting.
I20260810 00:57:38.978446 22662 tablet_bootstrap.cc:492] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152: No bootstrap required, opened a new log
I20260810 00:57:38.978554 22664 tablet_bootstrap.cc:654] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P de9db0ea984343c69e5890549cbcff64: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:38.978597 22662 ts_tablet_manager.cc:1397] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.001s
I20260810 00:57:38.979383 22662 raft_consensus.cc:348] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } }
I20260810 00:57:38.979974 22662 raft_consensus.cc:374] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:38.980129 22662 raft_consensus.cc:729] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 960a71f15b63450a8b0f706715a2f152, State: Initialized, Role: FOLLOWER
I20260810 00:57:38.980436 22662 consensus_queue.cc:260] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } }
I20260810 00:57:38.983637 22662 ts_tablet_manager.cc:1428] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260810 00:57:38.985236 22690 ts_tablet_manager.cc:1918] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:57:38.986189 22690 log.cc:1196] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/0a0a352416df4edc9c58c59d4dd4e103
I20260810 00:57:38.986775 22664 tablet_bootstrap.cc:492] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P de9db0ea984343c69e5890549cbcff64: No bootstrap required, opened a new log
I20260810 00:57:38.986802 22665 tablet_bootstrap.cc:492] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 21bed4c85c6249a5bbe628d811ae5e6c: No bootstrap required, opened a new log
I20260810 00:57:38.986905 22690 ts_tablet_manager.cc:1939] T 0a0a352416df4edc9c58c59d4dd4e103 P 21bed4c85c6249a5bbe628d811ae5e6c: Deleting consensus metadata
I20260810 00:57:38.986940 22664 ts_tablet_manager.cc:1397] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P de9db0ea984343c69e5890549cbcff64: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260810 00:57:38.986963 22665 ts_tablet_manager.cc:1397] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 21bed4c85c6249a5bbe628d811ae5e6c: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20260810 00:57:38.987766 22664 raft_consensus.cc:348] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P de9db0ea984343c69e5890549cbcff64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } }
I20260810 00:57:38.987995 22664 raft_consensus.cc:374] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P de9db0ea984343c69e5890549cbcff64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:38.987936 22665 raft_consensus.cc:348] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 21bed4c85c6249a5bbe628d811ae5e6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } }
I20260810 00:57:38.988090 22664 raft_consensus.cc:729] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P de9db0ea984343c69e5890549cbcff64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de9db0ea984343c69e5890549cbcff64, State: Initialized, Role: FOLLOWER
I20260810 00:57:38.988180 22665 raft_consensus.cc:374] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 21bed4c85c6249a5bbe628d811ae5e6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:38.988282 22665 raft_consensus.cc:729] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 21bed4c85c6249a5bbe628d811ae5e6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21bed4c85c6249a5bbe628d811ae5e6c, State: Initialized, Role: FOLLOWER
I20260810 00:57:38.988538 22665 consensus_queue.cc:260] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 21bed4c85c6249a5bbe628d811ae5e6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } }
I20260810 00:57:38.988538 22664 consensus_queue.cc:260] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P de9db0ea984343c69e5890549cbcff64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } }
I20260810 00:57:38.988788 22073 catalog_manager.cc:4841] TS 21bed4c85c6249a5bbe628d811ae5e6c (127.20.81.130:40029): tablet 0a0a352416df4edc9c58c59d4dd4e103 (table test-table [id=8a85b0ace4894b6b876330aaba936c6c]) successfully deleted
I20260810 00:57:38.989125 22665 ts_tablet_manager.cc:1428] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 21bed4c85c6249a5bbe628d811ae5e6c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 00:57:38.990393 22692 ts_tablet_manager.cc:1918] T 0a0a352416df4edc9c58c59d4dd4e103 P 960a71f15b63450a8b0f706715a2f152: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:57:38.990646 22692 log.cc:1196] T 0a0a352416df4edc9c58c59d4dd4e103 P 960a71f15b63450a8b0f706715a2f152: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/0a0a352416df4edc9c58c59d4dd4e103
I20260810 00:57:38.991235 22692 ts_tablet_manager.cc:1939] T 0a0a352416df4edc9c58c59d4dd4e103 P 960a71f15b63450a8b0f706715a2f152: Deleting consensus metadata
I20260810 00:57:38.991273 22664 ts_tablet_manager.cc:1428] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P de9db0ea984343c69e5890549cbcff64: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260810 00:57:38.993017 22074 catalog_manager.cc:4841] TS 960a71f15b63450a8b0f706715a2f152 (127.20.81.131:43717): tablet 0a0a352416df4edc9c58c59d4dd4e103 (table test-table [id=8a85b0ace4894b6b876330aaba936c6c]) successfully deleted
I20260810 00:57:39.016024 22669 raft_consensus.cc:482] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:57:39.016253 22669 raft_consensus.cc:504] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } }
I20260810 00:57:39.018059 22669 leader_election.cc:288] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de9db0ea984343c69e5890549cbcff64 (127.20.81.129:39081), 21bed4c85c6249a5bbe628d811ae5e6c (127.20.81.130:40029)
I20260810 00:57:39.026141 22661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:39.018599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:42965 (local address 127.20.81.129:39081)
0810 00:57:39.018787 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:39.018793 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:39.019600 (+ 807us) server_negotiation.cc:407] Connection header received
0810 00:57:39.019850 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:39.019855 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:39.019942 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:39.020070 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:57:39.020663 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.021532 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.023861 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.024971 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.025144 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:39.025223 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:39.025716 (+ 493us) server_negotiation.cc:299] Negotiation successful
0810 00:57:39.025844 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 00:57:39.026962 22658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:39.018575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:41329 (local address 127.20.81.130:40029)
0810 00:57:39.018794 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:39.018803 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:39.019429 (+ 626us) server_negotiation.cc:407] Connection header received
0810 00:57:39.019671 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:39.019676 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:39.019758 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:39.019853 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:57:39.020385 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.021605 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.024258 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.025607 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.025753 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:39.025835 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:39.026526 (+ 691us) server_negotiation.cc:299] Negotiation successful
0810 00:57:39.026662 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 00:57:39.027006 22697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:39.018310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:39081 (local address 127.20.81.131:42965)
0810 00:57:39.019482 (+ 1172us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:39.019501 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:39.019699 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:39.020118 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:39.020122 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:39.020143 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:57:39.020500 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:39.020508 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.021692 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.021703 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:39.023707 (+ 2004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:39.023720 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.025335 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.025342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:39.025515 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:39.025549 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 00:57:39.026391 (+ 842us) client_negotiation.cc:240] Negotiation successful
0810 00:57:39.026565 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1032,"mutex_wait_us":267,"thread_start_us":394,"threads_started":1}
I20260810 00:57:39.027078 22698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:39.018295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:40029 (local address 127.20.81.131:41329)
0810 00:57:39.019310 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:39.019330 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:39.019477 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:39.019875 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:39.019880 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:39.019922 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:57:39.020176 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:39.020189 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.021771 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.021777 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:39.024091 (+ 2314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:39.024108 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.025825 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.025832 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:39.026037 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:39.026137 (+ 100us) client_negotiation.cc:769] Sending connection context
0810 00:57:39.026498 (+ 361us) client_negotiation.cc:240] Negotiation successful
0810 00:57:39.026635 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":881,"mutex_wait_us":2,"spinlock_wait_cycles":3456,"thread_start_us":836,"threads_started":1}
I20260810 00:57:39.027823 22272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5d5c5e4fcb74edeb73c81db5d9d77f5" candidate_uuid: "960a71f15b63450a8b0f706715a2f152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de9db0ea984343c69e5890549cbcff64" is_pre_election: true
I20260810 00:57:39.028107 22272 raft_consensus.cc:2454] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P de9db0ea984343c69e5890549cbcff64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 960a71f15b63450a8b0f706715a2f152 in term 0.
I20260810 00:57:39.027994 22424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5d5c5e4fcb74edeb73c81db5d9d77f5" candidate_uuid: "960a71f15b63450a8b0f706715a2f152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" is_pre_election: true
I20260810 00:57:39.028532 22424 raft_consensus.cc:2454] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 21bed4c85c6249a5bbe628d811ae5e6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 960a71f15b63450a8b0f706715a2f152 in term 0.
I20260810 00:57:39.028672 22503 leader_election.cc:302] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 960a71f15b63450a8b0f706715a2f152, de9db0ea984343c69e5890549cbcff64; no voters:
I20260810 00:57:39.029088 22669 raft_consensus.cc:2781] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:57:39.029225 22669 raft_consensus.cc:482] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:57:39.029371 22669 raft_consensus.cc:3037] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:39.031831 22669 raft_consensus.cc:504] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } }
I20260810 00:57:39.032459 22669 leader_election.cc:288] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152 [CANDIDATE]: Term 1 election: Requested vote from peers de9db0ea984343c69e5890549cbcff64 (127.20.81.129:39081), 21bed4c85c6249a5bbe628d811ae5e6c (127.20.81.130:40029)
I20260810 00:57:39.032917 22272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5d5c5e4fcb74edeb73c81db5d9d77f5" candidate_uuid: "960a71f15b63450a8b0f706715a2f152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de9db0ea984343c69e5890549cbcff64"
I20260810 00:57:39.032984 22424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5d5c5e4fcb74edeb73c81db5d9d77f5" candidate_uuid: "960a71f15b63450a8b0f706715a2f152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21bed4c85c6249a5bbe628d811ae5e6c"
I20260810 00:57:39.033155 22272 raft_consensus.cc:3037] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P de9db0ea984343c69e5890549cbcff64 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:39.033205 22424 raft_consensus.cc:3037] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 21bed4c85c6249a5bbe628d811ae5e6c [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:39.035408 22272 raft_consensus.cc:2454] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P de9db0ea984343c69e5890549cbcff64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 960a71f15b63450a8b0f706715a2f152 in term 1.
I20260810 00:57:39.035808 22503 leader_election.cc:302] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 960a71f15b63450a8b0f706715a2f152, de9db0ea984343c69e5890549cbcff64; no voters:
I20260810 00:57:39.036098 22669 raft_consensus.cc:2781] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:57:39.036610 22424 raft_consensus.cc:2454] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 21bed4c85c6249a5bbe628d811ae5e6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 960a71f15b63450a8b0f706715a2f152 in term 1.
I20260810 00:57:39.036626 22669 raft_consensus.cc:686] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152 [term 1 LEADER]: Becoming Leader. State: Replica: 960a71f15b63450a8b0f706715a2f152, State: Running, Role: LEADER
I20260810 00:57:39.037006 22669 consensus_queue.cc:237] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } }
I20260810 00:57:39.040740 22088 catalog_manager.cc:5495] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 960a71f15b63450a8b0f706715a2f152 (127.20.81.131). New cstate: current_term: 1 leader_uuid: "960a71f15b63450a8b0f706715a2f152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } health_report { overall_health: HEALTHY } } }
Aug 10 00:57:39 dist-test-slave-gp6n krb5kdc[22039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323456, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:39.101485 22708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:39.081113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:36433 (local address 127.0.0.1:36968)
0810 00:57:39.082044 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:39.082065 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:39.082227 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:39.082680 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:39.082686 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:39.083661 (+ 975us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:39.084031 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:39.084044 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.085296 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.085301 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:39.086253 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:39.086265 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.087051 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.087056 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:39.087243 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:39.092255 (+ 5012us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:39.092308 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:39.095181 (+ 2873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:39.099071 (+ 3890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:39.099082 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:39.099088 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:39.099501 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:39.099919 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:39.099922 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:39.099925 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:39.100067 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:39.100658 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:39.100664 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:39.100825 (+ 161us) client_negotiation.cc:769] Sending connection context
0810 00:57:39.100958 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 00:57:39.101161 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"thread_start_us":599,"threads_started":1}
I20260810 00:57:39.101786 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:39.081107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36968 (local address 127.20.81.190:36433)
0810 00:57:39.081869 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:39.081875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:39.082162 (+ 287us) server_negotiation.cc:407] Connection header received
0810 00:57:39.082359 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:39.082363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:39.082490 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:39.082585 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:39.084241 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.085138 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.086435 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.086871 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.087019 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:39.095389 (+ 8370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:39.095396 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:39.095401 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:39.095442 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:39.098940 (+ 3498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:39.099618 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:39.099624 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:39.099628 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:39.099807 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:39.100165 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:39.100170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:39.100174 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:39.100529 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:39.100669 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:39.101082 (+ 413us) server_negotiation.cc:299] Negotiation successful
0810 00:57:39.101307 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":361,"threads_started":1}
I20260810 00:57:39.107473 22088 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:36968:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 00:57:39.108657 22088 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:57:39.115967 22243 tablet_service.cc:1467] Processing CreateTablet for tablet ce07b1b8000146cd9a28087a1728a3f2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f8991c7861e44f2d9861f16aca9617a9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 00:57:39.116358 22556 tablet_service.cc:1467] Processing CreateTablet for tablet ce07b1b8000146cd9a28087a1728a3f2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f8991c7861e44f2d9861f16aca9617a9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 00:57:39.116582 22243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce07b1b8000146cd9a28087a1728a3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:39.116449 22402 tablet_service.cc:1467] Processing CreateTablet for tablet ce07b1b8000146cd9a28087a1728a3f2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f8991c7861e44f2d9861f16aca9617a9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 00:57:39.116881 22556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce07b1b8000146cd9a28087a1728a3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:39.117013 22402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce07b1b8000146cd9a28087a1728a3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:39.124908 22662 tablet_bootstrap.cc:492] T ce07b1b8000146cd9a28087a1728a3f2 P 960a71f15b63450a8b0f706715a2f152: Bootstrap starting.
I20260810 00:57:39.125137 22664 tablet_bootstrap.cc:492] T ce07b1b8000146cd9a28087a1728a3f2 P de9db0ea984343c69e5890549cbcff64: Bootstrap starting.
I20260810 00:57:39.126683 22662 tablet_bootstrap.cc:654] T ce07b1b8000146cd9a28087a1728a3f2 P 960a71f15b63450a8b0f706715a2f152: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:39.126902 22664 tablet_bootstrap.cc:654] T ce07b1b8000146cd9a28087a1728a3f2 P de9db0ea984343c69e5890549cbcff64: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:39.127827 22665 tablet_bootstrap.cc:492] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c: Bootstrap starting.
I20260810 00:57:39.129626 22665 tablet_bootstrap.cc:654] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:39.136364 22664 tablet_bootstrap.cc:492] T ce07b1b8000146cd9a28087a1728a3f2 P de9db0ea984343c69e5890549cbcff64: No bootstrap required, opened a new log
I20260810 00:57:39.136632 22664 ts_tablet_manager.cc:1397] T ce07b1b8000146cd9a28087a1728a3f2 P de9db0ea984343c69e5890549cbcff64: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.001s
I20260810 00:57:39.137591 22665 tablet_bootstrap.cc:492] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c: No bootstrap required, opened a new log
I20260810 00:57:39.137636 22662 tablet_bootstrap.cc:492] T ce07b1b8000146cd9a28087a1728a3f2 P 960a71f15b63450a8b0f706715a2f152: No bootstrap required, opened a new log
I20260810 00:57:39.137537 22664 raft_consensus.cc:348] T ce07b1b8000146cd9a28087a1728a3f2 P de9db0ea984343c69e5890549cbcff64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } }
I20260810 00:57:39.137840 22662 ts_tablet_manager.cc:1397] T ce07b1b8000146cd9a28087a1728a3f2 P 960a71f15b63450a8b0f706715a2f152: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.001s
I20260810 00:57:39.137867 22664 raft_consensus.cc:374] T ce07b1b8000146cd9a28087a1728a3f2 P de9db0ea984343c69e5890549cbcff64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:39.137871 22665 ts_tablet_manager.cc:1397] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260810 00:57:39.137991 22664 raft_consensus.cc:729] T ce07b1b8000146cd9a28087a1728a3f2 P de9db0ea984343c69e5890549cbcff64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de9db0ea984343c69e5890549cbcff64, State: Initialized, Role: FOLLOWER
I20260810 00:57:39.138357 22664 consensus_queue.cc:260] T ce07b1b8000146cd9a28087a1728a3f2 P de9db0ea984343c69e5890549cbcff64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } }
I20260810 00:57:39.138659 22665 raft_consensus.cc:348] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } }
I20260810 00:57:39.138904 22665 raft_consensus.cc:374] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:39.139005 22665 raft_consensus.cc:729] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21bed4c85c6249a5bbe628d811ae5e6c, State: Initialized, Role: FOLLOWER
I20260810 00:57:39.138971 22672 tablet_replica.cc:439] T ce07b1b8000146cd9a28087a1728a3f2 P de9db0ea984343c69e5890549cbcff64: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } } }
I20260810 00:57:39.139328 22672 tablet_replica.cc:442] T ce07b1b8000146cd9a28087a1728a3f2 P de9db0ea984343c69e5890549cbcff64: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:57:39.139292 22665 consensus_queue.cc:260] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } }
I20260810 00:57:39.139771 22662 raft_consensus.cc:348] T ce07b1b8000146cd9a28087a1728a3f2 P 960a71f15b63450a8b0f706715a2f152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } }
I20260810 00:57:39.139730 22676 tablet_replica.cc:439] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } } }
I20260810 00:57:39.140026 22662 raft_consensus.cc:374] T ce07b1b8000146cd9a28087a1728a3f2 P 960a71f15b63450a8b0f706715a2f152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:39.140051 22676 tablet_replica.cc:442] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:57:39.140133 22662 raft_consensus.cc:729] T ce07b1b8000146cd9a28087a1728a3f2 P 960a71f15b63450a8b0f706715a2f152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 960a71f15b63450a8b0f706715a2f152, State: Initialized, Role: FOLLOWER
I20260810 00:57:39.140391 22662 consensus_queue.cc:260] T ce07b1b8000146cd9a28087a1728a3f2 P 960a71f15b63450a8b0f706715a2f152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } }
I20260810 00:57:39.140852 22669 tablet_replica.cc:439] T ce07b1b8000146cd9a28087a1728a3f2 P 960a71f15b63450a8b0f706715a2f152: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } } }
I20260810 00:57:39.141152 22669 tablet_replica.cc:442] T ce07b1b8000146cd9a28087a1728a3f2 P 960a71f15b63450a8b0f706715a2f152: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:57:39.141762 22662 ts_tablet_manager.cc:1428] T ce07b1b8000146cd9a28087a1728a3f2 P 960a71f15b63450a8b0f706715a2f152: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260810 00:57:39.142197 22665 ts_tablet_manager.cc:1428] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260810 00:57:39.142596 22664 ts_tablet_manager.cc:1428] T ce07b1b8000146cd9a28087a1728a3f2 P de9db0ea984343c69e5890549cbcff64: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260810 00:57:39.170799 22676 raft_consensus.cc:482] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:57:39.171093 22676 raft_consensus.cc:504] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } }
I20260810 00:57:39.171960 22676 leader_election.cc:288] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de9db0ea984343c69e5890549cbcff64 (127.20.81.129:39081), 960a71f15b63450a8b0f706715a2f152 (127.20.81.131:43717)
I20260810 00:57:39.172457 22272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce07b1b8000146cd9a28087a1728a3f2" candidate_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de9db0ea984343c69e5890549cbcff64" is_pre_election: true
I20260810 00:57:39.172540 22586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce07b1b8000146cd9a28087a1728a3f2" candidate_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "960a71f15b63450a8b0f706715a2f152" is_pre_election: true
I20260810 00:57:39.172760 22272 raft_consensus.cc:2454] T ce07b1b8000146cd9a28087a1728a3f2 P de9db0ea984343c69e5890549cbcff64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21bed4c85c6249a5bbe628d811ae5e6c in term 0.
I20260810 00:57:39.172844 22586 raft_consensus.cc:2454] T ce07b1b8000146cd9a28087a1728a3f2 P 960a71f15b63450a8b0f706715a2f152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21bed4c85c6249a5bbe628d811ae5e6c in term 0.
I20260810 00:57:39.173195 22348 leader_election.cc:302] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21bed4c85c6249a5bbe628d811ae5e6c, de9db0ea984343c69e5890549cbcff64; no voters:
I20260810 00:57:39.173481 22676 raft_consensus.cc:2781] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:57:39.173597 22676 raft_consensus.cc:482] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:57:39.173710 22676 raft_consensus.cc:3037] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:39.177016 22676 raft_consensus.cc:504] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } }
I20260810 00:57:39.177655 22676 leader_election.cc:288] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c [CANDIDATE]: Term 1 election: Requested vote from peers de9db0ea984343c69e5890549cbcff64 (127.20.81.129:39081), 960a71f15b63450a8b0f706715a2f152 (127.20.81.131:43717)
I20260810 00:57:39.178035 22272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce07b1b8000146cd9a28087a1728a3f2" candidate_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de9db0ea984343c69e5890549cbcff64"
I20260810 00:57:39.178257 22272 raft_consensus.cc:3037] T ce07b1b8000146cd9a28087a1728a3f2 P de9db0ea984343c69e5890549cbcff64 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:39.178201 22586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce07b1b8000146cd9a28087a1728a3f2" candidate_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "960a71f15b63450a8b0f706715a2f152"
I20260810 00:57:39.178431 22586 raft_consensus.cc:3037] T ce07b1b8000146cd9a28087a1728a3f2 P 960a71f15b63450a8b0f706715a2f152 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:39.180716 22586 raft_consensus.cc:2454] T ce07b1b8000146cd9a28087a1728a3f2 P 960a71f15b63450a8b0f706715a2f152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21bed4c85c6249a5bbe628d811ae5e6c in term 1.
I20260810 00:57:39.181141 22348 leader_election.cc:302] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21bed4c85c6249a5bbe628d811ae5e6c, 960a71f15b63450a8b0f706715a2f152; no voters:
I20260810 00:57:39.181334 22272 raft_consensus.cc:2454] T ce07b1b8000146cd9a28087a1728a3f2 P de9db0ea984343c69e5890549cbcff64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21bed4c85c6249a5bbe628d811ae5e6c in term 1.
I20260810 00:57:39.181414 22676 raft_consensus.cc:2781] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:57:39.181620 22676 raft_consensus.cc:686] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c [term 1 LEADER]: Becoming Leader. State: Replica: 21bed4c85c6249a5bbe628d811ae5e6c, State: Running, Role: LEADER
I20260810 00:57:39.182169 22676 consensus_queue.cc:237] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } }
I20260810 00:57:39.184091 22668 tablet_replica.cc:439] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c: TxnStatusTablet state changed. Reason: New leader 21bed4c85c6249a5bbe628d811ae5e6c. Latest consensus state: current_term: 1 leader_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } } }
I20260810 00:57:39.184366 22668 tablet_replica.cc:442] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:57:39.185245 22713 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 00:57:39.185626 22713 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 00:57:39.185719 22087 catalog_manager.cc:5495] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 21bed4c85c6249a5bbe628d811ae5e6c (127.20.81.130). New cstate: current_term: 1 leader_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } health_report { overall_health: UNKNOWN } } }
I20260810 00:57:39.216228 22658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:39.209828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54504 (local address 127.20.81.130:40029)
0810 00:57:39.210137 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:39.210148 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:39.210705 (+ 557us) server_negotiation.cc:407] Connection header received
0810 00:57:39.211019 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:39.211023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:39.211088 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:39.211165 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:39.211670 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.212456 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.214442 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.214899 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.215002 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:39.215514 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:39.215608 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:39.215673 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:39.215846 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 00:57:39.215912 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260810 00:57:39.216341 22708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:39.210215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:40029 (local address 127.0.0.1:54504)
0810 00:57:39.210596 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:39.210617 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:39.210780 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:39.211226 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:39.211233 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:39.211247 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:57:39.211450 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:39.211461 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.212594 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.212599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:39.214257 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:39.214274 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.215054 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.215060 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:39.215277 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:39.215346 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:57:39.215709 (+ 363us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:57:39.215728 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 00:57:39.215819 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 00:57:39.216019 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 00:57:39.220543 22586 raft_consensus.cc:1261] T ce07b1b8000146cd9a28087a1728a3f2 P 960a71f15b63450a8b0f706715a2f152 [term 1 FOLLOWER]: Refusing update from remote peer 21bed4c85c6249a5bbe628d811ae5e6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:57:39.220638 22272 raft_consensus.cc:1261] T ce07b1b8000146cd9a28087a1728a3f2 P de9db0ea984343c69e5890549cbcff64 [term 1 FOLLOWER]: Refusing update from remote peer 21bed4c85c6249a5bbe628d811ae5e6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:57:39.221413 22676 consensus_queue.cc:1035] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:57:39.222184 22668 consensus_queue.cc:1035] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:57:39.225466 22672 tablet_replica.cc:439] T ce07b1b8000146cd9a28087a1728a3f2 P de9db0ea984343c69e5890549cbcff64: TxnStatusTablet state changed. Reason: New leader 21bed4c85c6249a5bbe628d811ae5e6c. Latest consensus state: current_term: 1 leader_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } } }
I20260810 00:57:39.225783 22672 tablet_replica.cc:442] T ce07b1b8000146cd9a28087a1728a3f2 P de9db0ea984343c69e5890549cbcff64: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:57:39.228603 22672 tablet_replica.cc:439] T ce07b1b8000146cd9a28087a1728a3f2 P de9db0ea984343c69e5890549cbcff64: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } } }
I20260810 00:57:39.228842 22672 tablet_replica.cc:442] T ce07b1b8000146cd9a28087a1728a3f2 P de9db0ea984343c69e5890549cbcff64: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:57:39.231511 22668 tablet_replica.cc:439] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } } }
I20260810 00:57:39.231856 22668 tablet_replica.cc:442] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:57:39.235409 22669 tablet_replica.cc:439] T ce07b1b8000146cd9a28087a1728a3f2 P 960a71f15b63450a8b0f706715a2f152: TxnStatusTablet state changed. Reason: New leader 21bed4c85c6249a5bbe628d811ae5e6c. Latest consensus state: current_term: 1 leader_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } } }
I20260810 00:57:39.235728 22669 tablet_replica.cc:442] T ce07b1b8000146cd9a28087a1728a3f2 P 960a71f15b63450a8b0f706715a2f152: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:57:39.237062 22668 tablet_replica.cc:439] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } } }
I20260810 00:57:39.237370 22668 tablet_replica.cc:442] T ce07b1b8000146cd9a28087a1728a3f2 P 21bed4c85c6249a5bbe628d811ae5e6c: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:57:39.239593 22669 tablet_replica.cc:439] T ce07b1b8000146cd9a28087a1728a3f2 P 960a71f15b63450a8b0f706715a2f152: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 } } peers { permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 } } peers { permanent_uuid: "960a71f15b63450a8b0f706715a2f152" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43717 } } }
I20260810 00:57:39.239892 22669 tablet_replica.cc:442] T ce07b1b8000146cd9a28087a1728a3f2 P 960a71f15b63450a8b0f706715a2f152: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:57:39.244383 22656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:39.237628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38142 (local address 127.20.81.131:43717)
0810 00:57:39.237962 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:39.237968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:39.237985 (+ 17us) server_negotiation.cc:407] Connection header received
0810 00:57:39.238074 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:39.238079 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:39.238177 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:39.238352 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:39.238802 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.240134 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.242090 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.242637 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.242774 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:39.243276 (+ 502us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:39.243784 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:39.243951 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:39.244014 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 00:57:39.244094 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225}
I20260810 00:57:39.262271 22658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:39.255730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:39943 (local address 127.20.81.130:40029)
0810 00:57:39.255921 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:39.255926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:39.256488 (+ 562us) server_negotiation.cc:407] Connection header received
0810 00:57:39.256741 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:39.256747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:39.256830 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:39.256933 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:39.257416 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.258166 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.260187 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.260870 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.261002 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:39.261432 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:39.261595 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:39.261688 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:39.261896 (+ 208us) server_negotiation.cc:299] Negotiation successful
0810 00:57:39.261974 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 00:57:39.262341 22720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:39.255744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:40029 (local address 127.20.81.131:39943)
0810 00:57:39.256384 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:39.256400 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:39.256593 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:39.256944 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:39.256949 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:39.256961 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:57:39.257267 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:39.257276 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.258305 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.258312 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:39.260010 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:39.260023 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.261056 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.261061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:39.261193 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:39.261245 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:57:39.261738 (+ 493us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:57:39.261764 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 00:57:39.261846 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 00:57:39.261954 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":358,"threads_started":1}
I20260810 00:57:39.274555 22272 raft_consensus.cc:1261] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P de9db0ea984343c69e5890549cbcff64 [term 1 FOLLOWER]: Refusing update from remote peer 960a71f15b63450a8b0f706715a2f152: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:57:39.275592 22669 consensus_queue.cc:1035] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de9db0ea984343c69e5890549cbcff64" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:57:39.281826 22681 mvcc.cc:205] Tried to move back new op lower bound from 7316780889178558464 to 7316780888220127232. Current Snapshot: MvccSnapshot[applied={T|T < 7316780889178558464}]
I20260810 00:57:39.286194 22424 raft_consensus.cc:1261] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 21bed4c85c6249a5bbe628d811ae5e6c [term 1 FOLLOWER]: Refusing update from remote peer 960a71f15b63450a8b0f706715a2f152: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:57:39.288478 22669 consensus_queue.cc:1035] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21bed4c85c6249a5bbe628d811ae5e6c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:57:39.351266 22656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:39.332246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:51147 (local address 127.20.81.131:43717)
0810 00:57:39.332439 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:39.332454 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:39.343019 (+ 10565us) server_negotiation.cc:407] Connection header received
0810 00:57:39.343233 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:39.343238 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:39.343334 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:39.343428 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:39.346303 (+ 2875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.347435 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.349009 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.349703 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.349847 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:39.350364 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:39.350526 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:39.350648 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:39.350831 (+ 183us) server_negotiation.cc:299] Negotiation successful
0810 00:57:39.350912 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 00:57:39.351338 22729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:39.332182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:43717 (local address 127.20.81.130:51147)
0810 00:57:39.342895 (+ 10713us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:39.342912 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:39.343092 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:39.343605 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:39.343610 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:39.343623 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:57:39.346078 (+ 2455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:39.346094 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.347564 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.347569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:39.348863 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:39.348875 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.349913 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.349921 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:39.350109 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:39.350174 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:57:39.350694 (+ 520us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:57:39.350703 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 00:57:39.350799 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 00:57:39.350933 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10515,"thread_start_us":10364,"threads_started":1}
I20260810 00:57:39.398887 22087 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:57:39.399163 22087 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:57:39.402868 22087 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d5357d3a65f943fe87bac39fb22f9b6a: Sending DeleteTablet for 3 replicas of tablet a5d5c5e4fcb74edeb73c81db5d9d77f5
I20260810 00:57:39.404264 22556 tablet_service.cc:1514] Processing DeleteTablet for tablet a5d5c5e4fcb74edeb73c81db5d9d77f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:57:39 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:38136
I20260810 00:57:39.404270 22243 tablet_service.cc:1514] Processing DeleteTablet for tablet a5d5c5e4fcb74edeb73c81db5d9d77f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:57:39 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:39118
I20260810 00:57:39.404573 22693 tablet_replica.cc:330] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P de9db0ea984343c69e5890549cbcff64: stopping tablet replica
I20260810 00:57:39.404579 22692 tablet_replica.cc:330] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152: stopping tablet replica
I20260810 00:57:39.404791 22692 raft_consensus.cc:2229] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:57:39.404789 22693 raft_consensus.cc:2229] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P de9db0ea984343c69e5890549cbcff64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:57:39.404831 22402 tablet_service.cc:1514] Processing DeleteTablet for tablet a5d5c5e4fcb74edeb73c81db5d9d77f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:57:39 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:54480
I20260810 00:57:39.405004 22693 raft_consensus.cc:2258] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P de9db0ea984343c69e5890549cbcff64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:57:39.405107 22690 tablet_replica.cc:330] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 21bed4c85c6249a5bbe628d811ae5e6c: stopping tablet replica
I20260810 00:57:39.405296 22692 raft_consensus.cc:2258] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:57:39.405306 22690 raft_consensus.cc:2229] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 21bed4c85c6249a5bbe628d811ae5e6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:57:39.405516 22690 raft_consensus.cc:2258] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 21bed4c85c6249a5bbe628d811ae5e6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:57:39.406812 22693 ts_tablet_manager.cc:1905] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P de9db0ea984343c69e5890549cbcff64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:57:39.407064 22690 ts_tablet_manager.cc:1905] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 21bed4c85c6249a5bbe628d811ae5e6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:57:39.410291 22692 ts_tablet_manager.cc:1905] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:57:39.417773 22690 ts_tablet_manager.cc:1918] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 21bed4c85c6249a5bbe628d811ae5e6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:57:39.417981 22690 log.cc:1196] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 21bed4c85c6249a5bbe628d811ae5e6c: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/a5d5c5e4fcb74edeb73c81db5d9d77f5
I20260810 00:57:39.421937 22690 ts_tablet_manager.cc:1939] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 21bed4c85c6249a5bbe628d811ae5e6c: Deleting consensus metadata
I20260810 00:57:39.422971 22693 ts_tablet_manager.cc:1918] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P de9db0ea984343c69e5890549cbcff64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:57:39.423208 22693 log.cc:1196] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P de9db0ea984343c69e5890549cbcff64: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-0/wal/wals/a5d5c5e4fcb74edeb73c81db5d9d77f5
I20260810 00:57:39.424070 22073 catalog_manager.cc:4841] TS 21bed4c85c6249a5bbe628d811ae5e6c (127.20.81.130:40029): tablet a5d5c5e4fcb74edeb73c81db5d9d77f5 (table test-table [id=223e825673114643b26c11c110b83512]) successfully deleted
I20260810 00:57:39.424757 22693 ts_tablet_manager.cc:1939] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P de9db0ea984343c69e5890549cbcff64: Deleting consensus metadata
I20260810 00:57:39.425001 22692 ts_tablet_manager.cc:1918] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:57:39.426182 22074 catalog_manager.cc:4841] TS de9db0ea984343c69e5890549cbcff64 (127.20.81.129:39081): tablet a5d5c5e4fcb74edeb73c81db5d9d77f5 (table test-table [id=223e825673114643b26c11c110b83512]) successfully deleted
I20260810 00:57:39.426540 22692 log.cc:1196] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/a5d5c5e4fcb74edeb73c81db5d9d77f5
I20260810 00:57:39.427459 22692 ts_tablet_manager.cc:1939] T a5d5c5e4fcb74edeb73c81db5d9d77f5 P 960a71f15b63450a8b0f706715a2f152: Deleting consensus metadata
I20260810 00:57:39.428620 22074 catalog_manager.cc:4841] TS 960a71f15b63450a8b0f706715a2f152 (127.20.81.131:43717): tablet a5d5c5e4fcb74edeb73c81db5d9d77f5 (table test-table [id=223e825673114643b26c11c110b83512]) successfully deleted
Aug 10 00:57:39 dist-test-slave-gp6n krb5kdc[22039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323458, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.81.129@KRBTEST.COM
I20260810 00:57:39.448689 22661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:39.426775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39124 (local address 127.20.81.129:39081)
0810 00:57:39.426953 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:39.426959 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:39.429578 (+ 2619us) server_negotiation.cc:407] Connection header received
0810 00:57:39.429760 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:39.429766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:39.429865 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:39.429980 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:39.431999 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.433131 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.435069 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.435703 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.435858 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:39.441113 (+ 5255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:39.441135 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:39.441159 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:39.441272 (+ 113us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:39.445271 (+ 3999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:39.446183 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:39.446193 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:39.446197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:39.446371 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:39.446904 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:39.446911 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:39.446916 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:39.447425 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:39.447751 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:39.448079 (+ 328us) server_negotiation.cc:299] Negotiation successful
0810 00:57:39.448302 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260810 00:57:39.449483 22315 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39124
I20260810 00:57:39.471963 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:39.455651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36972 (local address 127.20.81.190:36433)
0810 00:57:39.455854 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:39.455860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:39.456182 (+ 322us) server_negotiation.cc:407] Connection header received
0810 00:57:39.456357 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:39.456363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:39.456455 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:39.456550 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:39.458240 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.459404 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.461447 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.462106 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.462237 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:39.465363 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:39.465370 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:39.465377 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:39.465422 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:39.469033 (+ 3611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:39.469858 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:39.469865 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:39.469869 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:39.470023 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:39.470466 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:39.470470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:39.470473 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:39.470799 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:39.471043 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:39.471408 (+ 365us) server_negotiation.cc:299] Negotiation successful
0810 00:57:39.471623 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260810 00:57:39.472855 22087 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36972
I20260810 00:57:39.478935 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 22168
I20260810 00:57:39.506564 22661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:39.481718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39136 (local address 127.20.81.129:39081)
0810 00:57:39.481891 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:39.481897 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:39.490909 (+ 9012us) server_negotiation.cc:407] Connection header received
0810 00:57:39.491129 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:39.491136 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:39.491223 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:39.491299 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:39.493010 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.494166 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.496005 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.496643 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.496730 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:39.499818 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:39.499825 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:39.499829 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:39.499872 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:39.503328 (+ 3456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:39.504140 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:39.504146 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:39.504150 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:39.504286 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:39.504705 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:39.504710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:39.504720 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:39.505108 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:39.505383 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:39.505954 (+ 571us) server_negotiation.cc:299] Negotiation successful
0810 00:57:39.506170 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260810 00:57:39.508322 22315 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39136
W20260810 00:57:39.509027 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 22168: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:57:39.509225 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 22168
I20260810 00:57:39.535254 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 22326
Aug 10 00:57:39 dist-test-slave-gp6n krb5kdc[22039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323458, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.81.130@KRBTEST.COM
I20260810 00:57:39.553864 22658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:39.537038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54514 (local address 127.20.81.130:40029)
0810 00:57:39.537223 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:39.537228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:39.537247 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:57:39.537318 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:39.537323 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:39.537405 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:39.537507 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:39.539353 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.540549 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.542544 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.543110 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.543257 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:39.547645 (+ 4388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:39.547659 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:39.547676 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:39.547708 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:39.550870 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:39.551594 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:39.551601 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:39.551603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:39.551771 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:39.552214 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:39.552219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:39.552222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:39.552609 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:39.552905 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:39.553241 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 00:57:39.553488 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260810 00:57:39.554674 22471 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54514
W20260810 00:57:39.555395 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 22326: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:57:39.555585 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 22326
I20260810 00:57:39.587999 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 22482
Aug 10 00:57:39 dist-test-slave-gp6n krb5kdc[22039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323458, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.81.131@KRBTEST.COM
I20260810 00:57:39.606154 22656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:39.588812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38152 (local address 127.20.81.131:43717)
0810 00:57:39.589048 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:39.589053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:39.589076 (+ 23us) server_negotiation.cc:407] Connection header received
0810 00:57:39.589298 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:39.589302 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:39.589406 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:39.589497 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:39.591174 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.592002 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.593896 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:39.594635 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:39.594748 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:39.599839 (+ 5091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:39.599853 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:39.599869 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:39.599901 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:39.603359 (+ 3458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:39.604075 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:39.604082 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:39.604085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:39.604225 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:39.604634 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:39.604640 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:39.604644 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:39.605003 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:39.605424 (+ 421us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:39.605575 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 00:57:39.605798 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
W20260810 00:57:39.606858 22634 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38152
W20260810 00:57:39.607527 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 22482: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:57:39.607699 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 22482
I20260810 00:57:39.627079 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 22054
I20260810 00:57:39.628336 22134 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:57:39.743238 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 22054
2026-08-10T00:57:39Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5266 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:57:39 dist-test-slave-gp6n krb5kdc[22743](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:57:39 dist-test-slave-gp6n krb5kdc[22743](info): set up 2 sockets
Aug 10 00:57:39 dist-test-slave-gp6n krb5kdc[22743](info): commencing operation
krb5kdc: starting...
W20260810 00:57:41.001329 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 00:57:41 dist-test-slave-gp6n krb5kdc[22743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:57:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:57:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 00:57:41.218257 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:38411
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:38411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:41.548292 22758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:41.548652 22758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:41.548826 22758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:41.555904 22758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:57:41.556047 22758 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 00:57:41.556129 22758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:41.556226 22758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:41.556324 22758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:57:41.556463 22758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:57:41.566098 22758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:38411
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:38411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:41.567416 22758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:41.568745 22758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:41.580539 22763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:41.580549 22764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:41.580579 22766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:41.581383 22758 server_base.cc:1034] running on GCE node
I20260810 00:57:41.582065 22758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:41.583437 22758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:41.584661 22758 hybrid_clock.cc:648] HybridClock initialized: now 1786323461584608 us; error 25 us; skew 500 ppm
I20260810 00:57:41.584940 22758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:41 dist-test-slave-gp6n krb5kdc[22743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323461, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:41.588251 22758 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 00:57:41.589658 22758 webserver.cc:460] Webserver started at http://127.20.81.190:41819/ using document root <none> and password file <none>
I20260810 00:57:41.590327 22758 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:41.590560 22758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:41.590955 22758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:41.594269 22758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "c23651cd115445f8ac5de966480c07d0"
format_stamp: "Formatted at 2026-08-10 00:57:41 on dist-test-slave-gp6n"
server_key: "b9e719b254f33416138e8ca516ed650c"
server_key_iv: "46da98870244ff43b54ddfb4c5b3e373"
server_key_version: "encryptionkey@0"
I20260810 00:57:41.595162 22758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "c23651cd115445f8ac5de966480c07d0"
format_stamp: "Formatted at 2026-08-10 00:57:41 on dist-test-slave-gp6n"
server_key: "b9e719b254f33416138e8ca516ed650c"
server_key_iv: "46da98870244ff43b54ddfb4c5b3e373"
server_key_version: "encryptionkey@0"
I20260810 00:57:41.600716 22758 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 00:57:41.604210 22775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:41.605558 22758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 00:57:41.605697 22758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "c23651cd115445f8ac5de966480c07d0"
format_stamp: "Formatted at 2026-08-10 00:57:41 on dist-test-slave-gp6n"
server_key: "b9e719b254f33416138e8ca516ed650c"
server_key_iv: "46da98870244ff43b54ddfb4c5b3e373"
server_key_version: "encryptionkey@0"
I20260810 00:57:41.605883 22758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:41.632715 22758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:41.637403 22758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:41.637624 22758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:41.660105 22758 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:38411
I20260810 00:57:41.660126 22843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:38411 every 8 connection(s)
I20260810 00:57:41.661494 22758 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:57:41.665207 22844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:41.668267 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 22758
I20260810 00:57:41.668512 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 00:57:41.669256 20806 external_mini_cluster.cc:1428] Setting key 93cd33987ed91e3c39a4a68f3cc74f26
I20260810 00:57:41.677412 22844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0: Bootstrap starting.
I20260810 00:57:41.680481 22844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:41.681543 22844 log.cc:824] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0: Log is configured to *not* fsync() on all Append() calls
Aug 10 00:57:41 dist-test-slave-gp6n krb5kdc[22743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:41.684275 22844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0: No bootstrap required, opened a new log
I20260810 00:57:41.692153 22844 raft_consensus.cc:348] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c23651cd115445f8ac5de966480c07d0" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38411 } }
I20260810 00:57:41.692543 22844 raft_consensus.cc:374] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:41.692683 22844 raft_consensus.cc:729] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c23651cd115445f8ac5de966480c07d0, State: Initialized, Role: FOLLOWER
I20260810 00:57:41.693300 22844 consensus_queue.cc:260] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c23651cd115445f8ac5de966480c07d0" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38411 } }
I20260810 00:57:41.693468 22847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:41.671458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41086 (local address 127.20.81.190:38411)
0810 00:57:41.672285 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:41.672298 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:41.672343 (+ 45us) server_negotiation.cc:407] Connection header received
0810 00:57:41.674103 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:41.674126 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:41.674655 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:41.675136 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:41.676704 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:41.678249 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:41.679623 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:41.680365 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:41.680595 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:41.684995 (+ 4400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:41.685003 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:41.685019 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:41.685054 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:41.688675 (+ 3621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:41.689457 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:41.689464 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:41.689471 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:41.689657 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:41.690112 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:41.690115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:41.690118 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:41.690782 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:41.691143 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:41.691652 (+ 509us) server_negotiation.cc:299] Negotiation successful
0810 00:57:41.692086 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":393,"threads_started":1}
I20260810 00:57:41.693579 22844 raft_consensus.cc:388] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:57:41.693770 22844 raft_consensus.cc:482] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:57:41.693959 22844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:41.696655 22844 raft_consensus.cc:504] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c23651cd115445f8ac5de966480c07d0" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38411 } }
I20260810 00:57:41.697088 22844 leader_election.cc:302] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c23651cd115445f8ac5de966480c07d0; no voters:
I20260810 00:57:41.697621 22844 leader_election.cc:288] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:57:41.697706 22849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:57:41.698482 22849 raft_consensus.cc:686] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0 [term 1 LEADER]: Becoming Leader. State: Replica: c23651cd115445f8ac5de966480c07d0, State: Running, Role: LEADER
I20260810 00:57:41.699019 22849 consensus_queue.cc:237] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c23651cd115445f8ac5de966480c07d0" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38411 } }
I20260810 00:57:41.699656 22844 sys_catalog.cc:564] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:57:41.702057 22851 sys_catalog.cc:455] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c23651cd115445f8ac5de966480c07d0. Latest consensus state: current_term: 1 leader_uuid: "c23651cd115445f8ac5de966480c07d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c23651cd115445f8ac5de966480c07d0" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38411 } } }
I20260810 00:57:41.702101 22850 sys_catalog.cc:455] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c23651cd115445f8ac5de966480c07d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c23651cd115445f8ac5de966480c07d0" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38411 } } }
I20260810 00:57:41.702500 22851 sys_catalog.cc:458] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0 [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:41.702574 22850 sys_catalog.cc:458] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0 [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:41.703454 22855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:57:41.709817 22855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:57:41.721320 22855 catalog_manager.cc:1347] Generated new cluster ID: ec221ebc745648d7bfa7230eb961ee16
I20260810 00:57:41.721441 22855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:57:41.740726 22855 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:57:41.742061 22855 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:57:41.753533 22855 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0: Generated new TSK 0
I20260810 00:57:41.754388 22855 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.81.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
I20260810 00:57:41.814531 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:38411
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:42.146250 22874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:42.146572 22874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:42.146759 22874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:42.154132 22874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:42.154249 22874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:42.154456 22874 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 00:57:42.154546 22874 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 00:57:42.154676 22874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 00:57:42.164606 22874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.81.190:38411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:42.166251 22874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:42.167727 22874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:42.180967 22881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:42.181031 22880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:42.181008 22884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:42.182217 22874 server_base.cc:1034] running on GCE node
I20260810 00:57:42.182863 22874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:42.183835 22874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:42.185041 22874 hybrid_clock.cc:648] HybridClock initialized: now 1786323462185000 us; error 26 us; skew 500 ppm
I20260810 00:57:42.185389 22874 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:42 dist-test-slave-gp6n krb5kdc[22743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323462, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:42.188814 22874 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 00:57:42.190347 22874 webserver.cc:460] Webserver started at http://127.20.81.129:37571/ using document root <none> and password file <none>
I20260810 00:57:42.191052 22874 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:42.191202 22874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:42.191538 22874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:42.194698 22874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "492165fc756d4ec7a9b0d43f9624f6dc"
format_stamp: "Formatted at 2026-08-10 00:57:42 on dist-test-slave-gp6n"
server_key: "9ede77f303df9bd67af20dd1bae5cd70"
server_key_iv: "775384c62c63c57fb39d68215d8ad55b"
server_key_version: "encryptionkey@0"
I20260810 00:57:42.195468 22874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "492165fc756d4ec7a9b0d43f9624f6dc"
format_stamp: "Formatted at 2026-08-10 00:57:42 on dist-test-slave-gp6n"
server_key: "9ede77f303df9bd67af20dd1bae5cd70"
server_key_iv: "775384c62c63c57fb39d68215d8ad55b"
server_key_version: "encryptionkey@0"
I20260810 00:57:42.200646 22874 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 00:57:42.204211 22891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:42.205633 22874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 00:57:42.205799 22874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "492165fc756d4ec7a9b0d43f9624f6dc"
format_stamp: "Formatted at 2026-08-10 00:57:42 on dist-test-slave-gp6n"
server_key: "9ede77f303df9bd67af20dd1bae5cd70"
server_key_iv: "775384c62c63c57fb39d68215d8ad55b"
server_key_version: "encryptionkey@0"
I20260810 00:57:42.206014 22874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:42.231024 22874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:42.235968 22874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:42.236287 22874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:42.239260 22874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:57:42.239476 22874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260810 00:57:42.239645 22874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:57:42.239753 22874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 00:57:42 dist-test-slave-gp6n krb5kdc[22743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323462, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:42.265964 22900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:42.239570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:40807 (local address 127.20.81.190:38411)
0810 00:57:42.240158 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:42.240168 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:42.241500 (+ 1332us) server_negotiation.cc:407] Connection header received
0810 00:57:42.244797 (+ 3297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:42.244802 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:42.244866 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:42.245030 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:42.247856 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:42.248621 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:42.252490 (+ 3869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:42.253184 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:42.253355 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:42.258703 (+ 5348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:42.258707 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:42.258711 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:42.258744 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:42.262078 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:42.263105 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:42.263113 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:42.263117 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:42.263316 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:42.264107 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:42.264113 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:42.264117 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:42.264416 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:42.264597 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:42.265354 (+ 757us) server_negotiation.cc:299] Negotiation successful
0810 00:57:42.265578 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":272,"threads_started":1}
I20260810 00:57:42.267330 22902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:42.240175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38411 (local address 127.20.81.129:40807)
0810 00:57:42.241283 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:42.241352 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:42.244458 (+ 3106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:42.245348 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:42.245366 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:42.246443 (+ 1077us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:42.247638 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:42.247677 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:42.248886 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:42.248893 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:42.252270 (+ 3377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:42.252291 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:42.253944 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:42.253955 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:42.254292 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:42.255313 (+ 1021us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:42.255349 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:42.258382 (+ 3033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:42.262227 (+ 3845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:42.262236 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:42.262252 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:42.262970 (+ 718us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:42.263462 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:42.263475 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:42.263483 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:42.263907 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:42.264556 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:42.264567 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:42.264979 (+ 412us) client_negotiation.cc:769] Sending connection context
0810 00:57:42.265247 (+ 268us) client_negotiation.cc:240] Negotiation successful
0810 00:57:42.265790 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"thread_start_us":465,"threads_started":1}
I20260810 00:57:42.288214 22900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:42.273038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:43237 (local address 127.20.81.190:38411)
0810 00:57:42.273215 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:42.273222 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:42.274033 (+ 811us) server_negotiation.cc:407] Connection header received
0810 00:57:42.274241 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:42.274246 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:42.274313 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:42.274413 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:42.275378 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:42.276064 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:42.277348 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:42.277744 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:42.277887 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:42.280165 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:42.280175 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:42.280184 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:42.280236 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:42.284529 (+ 4293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:42.285581 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:42.285589 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:42.285596 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:42.285809 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:42.286316 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:42.286325 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:42.286332 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:42.286851 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:42.287036 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:42.287782 (+ 746us) server_negotiation.cc:299] Negotiation successful
0810 00:57:42.287971 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 00:57:42.288905 22966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:42.273221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38411 (local address 127.20.81.129:43237)
0810 00:57:42.273929 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:42.273946 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:42.274105 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:42.274483 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:42.274490 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:42.274960 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:42.275242 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:42.275250 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:42.276207 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:42.276218 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:42.277204 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:42.277214 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:42.277938 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:42.277945 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:42.278112 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:42.278871 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:42.278901 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:42.279911 (+ 1010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:42.284775 (+ 4864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:42.284783 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:42.284790 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:42.285419 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:42.285960 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:42.285967 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:42.285977 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:42.286164 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:42.287126 (+ 962us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:42.287135 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:42.287420 (+ 285us) client_negotiation.cc:769] Sending connection context
0810 00:57:42.287661 (+ 241us) client_negotiation.cc:240] Negotiation successful
0810 00:57:42.288186 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":443,"threads_started":1}
I20260810 00:57:42.300128 22874 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:39145
I20260810 00:57:42.300179 23029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:39145 every 8 connection(s)
I20260810 00:57:42.301419 22874 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 00:57:42.303623 23030 heartbeater.cc:347] Connected to a master server at 127.20.81.190:38411
I20260810 00:57:42.303872 23030 heartbeater.cc:464] Registering TS with master...
I20260810 00:57:42.304713 23030 heartbeater.cc:511] Master 127.20.81.190:38411 requested a full tablet report, sending...
I20260810 00:57:42.305837 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 22874
I20260810 00:57:42.306062 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 00:57:42.306836 20806 external_mini_cluster.cc:1428] Setting key b4f45dd929f5b1fc50d827fb90cfe75a
I20260810 00:57:42.307245 22793 ts_manager.cc:194] Registered new tserver with Master: 492165fc756d4ec7a9b0d43f9624f6dc (127.20.81.129:39145)
I20260810 00:57:42.309319 22793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:40807
WARNING: no policy specified for kudu/127.20.81.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
I20260810 00:57:42.354174 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:38411
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:42.705922 23035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:42.706216 23035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:42.706393 23035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:42.715167 23035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:42.715307 23035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:42.715498 23035 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 00:57:42.715588 23035 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 00:57:42.715713 23035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 00:57:42.726743 23035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.81.190:38411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:42.728138 23035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:42.729638 23035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:42.743424 23043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:42.743621 23045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:42.743680 23041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:42.744992 23035 server_base.cc:1034] running on GCE node
I20260810 00:57:42.745618 23035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:42.746577 23035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:42.747777 23035 hybrid_clock.cc:648] HybridClock initialized: now 1786323462747736 us; error 33 us; skew 500 ppm
I20260810 00:57:42.748062 23035 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:42 dist-test-slave-gp6n krb5kdc[22743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323462, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:42.751252 23035 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 00:57:42.752624 23035 webserver.cc:460] Webserver started at http://127.20.81.130:40323/ using document root <none> and password file <none>
I20260810 00:57:42.753275 23035 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:42.753384 23035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:42.753664 23035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:42.756662 23035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "872da1bf65c242c6a9e85b515e34cb9f"
format_stamp: "Formatted at 2026-08-10 00:57:42 on dist-test-slave-gp6n"
server_key: "ca66af40fe086aa5c38a4885556c4af6"
server_key_iv: "d5b4d84bb7e04d9435c2d37acef676ec"
server_key_version: "encryptionkey@0"
I20260810 00:57:42.757385 23035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "872da1bf65c242c6a9e85b515e34cb9f"
format_stamp: "Formatted at 2026-08-10 00:57:42 on dist-test-slave-gp6n"
server_key: "ca66af40fe086aa5c38a4885556c4af6"
server_key_iv: "d5b4d84bb7e04d9435c2d37acef676ec"
server_key_version: "encryptionkey@0"
I20260810 00:57:42.762594 23035 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 00:57:42.766021 23054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:42.767416 23035 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:57:42.767537 23035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "872da1bf65c242c6a9e85b515e34cb9f"
format_stamp: "Formatted at 2026-08-10 00:57:42 on dist-test-slave-gp6n"
server_key: "ca66af40fe086aa5c38a4885556c4af6"
server_key_iv: "d5b4d84bb7e04d9435c2d37acef676ec"
server_key_version: "encryptionkey@0"
I20260810 00:57:42.767699 23035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:42.782019 23035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:42.786478 23035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:42.786728 23035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:42.789535 23035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:57:42.789641 23035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:42.789796 23035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:57:42.789894 23035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 10 00:57:42 dist-test-slave-gp6n krb5kdc[22743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323462, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:42.815568 23064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:42.790146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:38633 (local address 127.20.81.190:38411)
0810 00:57:42.793009 (+ 2863us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:42.793014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:42.793034 (+ 20us) server_negotiation.cc:407] Connection header received
0810 00:57:42.795566 (+ 2532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:42.795576 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:42.795686 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:42.795829 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:42.798594 (+ 2765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:42.799556 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:42.802204 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:42.802882 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:42.803027 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:42.808269 (+ 5242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:42.808274 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:42.808278 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:42.808312 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:42.811670 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:42.812768 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:42.812777 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:42.812784 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:42.813002 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:42.813442 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:42.813447 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:42.813450 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:42.813904 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:42.814129 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:42.814935 (+ 806us) server_negotiation.cc:299] Negotiation successful
0810 00:57:42.815172 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2756,"thread_start_us":2646,"threads_started":1}
I20260810 00:57:42.816676 23065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:42.790986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38411 (local address 127.20.81.130:38633)
0810 00:57:42.792016 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:42.792073 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:42.795165 (+ 3092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:42.796182 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:42.796198 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:42.797259 (+ 1061us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:42.798367 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:42.798423 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:42.799840 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:42.799844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:42.802025 (+ 2181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:42.802038 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:42.803305 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:42.803310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:42.803513 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:42.804479 (+ 966us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:42.804558 (+ 79us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:42.808019 (+ 3461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:42.811826 (+ 3807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:42.811836 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:42.811860 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:42.812592 (+ 732us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:42.813137 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:42.813140 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:42.813142 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:42.813319 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:42.814050 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:42.814056 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:42.814478 (+ 422us) client_negotiation.cc:769] Sending connection context
0810 00:57:42.814842 (+ 364us) client_negotiation.cc:240] Negotiation successful
0810 00:57:42.815409 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"thread_start_us":446,"threads_started":1}
I20260810 00:57:42.839429 23064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:42.825316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:34981 (local address 127.20.81.190:38411)
0810 00:57:42.825554 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:42.825561 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:42.826080 (+ 519us) server_negotiation.cc:407] Connection header received
0810 00:57:42.826242 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:42.826250 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:42.826357 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:42.826513 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:42.828000 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:42.829037 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:42.830379 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:42.830952 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:42.831085 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:42.833606 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:42.833614 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:42.833620 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:42.833664 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:42.836754 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:42.837412 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:42.837419 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:42.837422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:42.837544 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:42.837907 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:42.837913 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:42.837924 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:42.838246 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:42.838397 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:42.838859 (+ 462us) server_negotiation.cc:299] Negotiation successful
0810 00:57:42.839088 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 00:57:42.839593 23128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:42.825279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38411 (local address 127.20.81.130:34981)
0810 00:57:42.825974 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:42.826000 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:42.826120 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:42.826553 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:42.826561 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:42.827160 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:42.827829 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:42.827842 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:42.829179 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:42.829188 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:42.830228 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:42.830241 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:42.831329 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:42.831334 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:42.831466 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:42.832162 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:42.832191 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:42.833325 (+ 1134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:42.836888 (+ 3563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:42.836893 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:42.836898 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:42.837292 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:42.837664 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:42.837668 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:42.837672 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:42.837800 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:42.838375 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:42.838380 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:42.838613 (+ 233us) client_negotiation.cc:769] Sending connection context
0810 00:57:42.838774 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 00:57:42.839128 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":391,"threads_started":1}
I20260810 00:57:42.854681 23035 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:41307
I20260810 00:57:42.854722 23189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:41307 every 8 connection(s)
I20260810 00:57:42.855933 23035 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 00:57:42.857952 23190 heartbeater.cc:347] Connected to a master server at 127.20.81.190:38411
I20260810 00:57:42.858212 23190 heartbeater.cc:464] Registering TS with master...
I20260810 00:57:42.858359 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 23035
I20260810 00:57:42.858569 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 00:57:42.859042 23190 heartbeater.cc:511] Master 127.20.81.190:38411 requested a full tablet report, sending...
I20260810 00:57:42.859192 20806 external_mini_cluster.cc:1428] Setting key e04c856ad422408fe9a062af7f4660dc
I20260810 00:57:42.860556 22793 ts_manager.cc:194] Registered new tserver with Master: 872da1bf65c242c6a9e85b515e34cb9f (127.20.81.130:41307)
I20260810 00:57:42.861406 22793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:38633
WARNING: no policy specified for kudu/127.20.81.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
I20260810 00:57:42.906251 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:38411
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:43.243523 23194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:43.243813 23194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:43.243980 23194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:43.251818 23194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:43.251924 23194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:43.252099 23194 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 00:57:43.252161 23194 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 00:57:43.252324 23194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 00:57:43.262099 23194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.81.190:38411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:43.263453 23194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:43.264822 23194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:43.278687 23201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:43.278759 23200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:43.278785 23204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:43.279595 23194 server_base.cc:1034] running on GCE node
I20260810 00:57:43.280133 23194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:43.281050 23194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:43.282261 23194 hybrid_clock.cc:648] HybridClock initialized: now 1786323463282222 us; error 38 us; skew 500 ppm
I20260810 00:57:43.282612 23194 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:43 dist-test-slave-gp6n krb5kdc[22743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323463, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:43.285907 23194 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 00:57:43.287393 23194 webserver.cc:460] Webserver started at http://127.20.81.131:36213/ using document root <none> and password file <none>
I20260810 00:57:43.288071 23194 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:43.288219 23194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:43.288538 23194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:43.291700 23194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "294011f87b014d1682732a8b7dc02f0c"
format_stamp: "Formatted at 2026-08-10 00:57:43 on dist-test-slave-gp6n"
server_key: "3ea0c2aa5f0b94b8afef5a6e248b1909"
server_key_iv: "9b3b9f476bd4c0c56d60ee13461a9f22"
server_key_version: "encryptionkey@0"
I20260810 00:57:43.292469 23194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "294011f87b014d1682732a8b7dc02f0c"
format_stamp: "Formatted at 2026-08-10 00:57:43 on dist-test-slave-gp6n"
server_key: "3ea0c2aa5f0b94b8afef5a6e248b1909"
server_key_iv: "9b3b9f476bd4c0c56d60ee13461a9f22"
server_key_version: "encryptionkey@0"
I20260810 00:57:43.297542 23194 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 00:57:43.301256 23211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:43.302763 23194 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 00:57:43.302927 23194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "294011f87b014d1682732a8b7dc02f0c"
format_stamp: "Formatted at 2026-08-10 00:57:43 on dist-test-slave-gp6n"
server_key: "3ea0c2aa5f0b94b8afef5a6e248b1909"
server_key_iv: "9b3b9f476bd4c0c56d60ee13461a9f22"
server_key_version: "encryptionkey@0"
I20260810 00:57:43.303105 23194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:43.312538 23030 heartbeater.cc:503] Master 127.20.81.190:38411 was elected leader, sending a full tablet report...
I20260810 00:57:43.328299 23194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:43.332842 23194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:43.333155 23194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:43.336051 23194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:57:43.336176 23194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:43.336308 23194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:57:43.336422 23194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 00:57:43 dist-test-slave-gp6n krb5kdc[22743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323463, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:43.361203 23064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.336583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:44445 (local address 127.20.81.190:38411)
0810 00:57:43.336786 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:43.336794 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:43.338251 (+ 1457us) server_negotiation.cc:407] Connection header received
0810 00:57:43.340401 (+ 2150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:43.340406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:43.340515 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:43.340629 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:43.342990 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.344045 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.346873 (+ 2828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.347590 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.347753 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.352664 (+ 4911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:43.352674 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:43.352681 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:43.352732 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:43.357126 (+ 4394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:43.357966 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:43.357974 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:43.357980 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:43.358179 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:43.358727 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:43.358731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:43.358733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:43.359171 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:43.359341 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:43.360458 (+ 1117us) server_negotiation.cc:299] Negotiation successful
0810 00:57:43.360729 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 00:57:43.362782 23221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.337248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38411 (local address 127.20.81.131:44445)
0810 00:57:43.338083 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:43.338125 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:43.340070 (+ 1945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:43.340883 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:43.340896 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:43.341816 (+ 920us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:43.342795 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.342824 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.344198 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.344206 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.346661 (+ 2455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.346673 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.347894 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.347904 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.348120 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.348886 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:43.348928 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:43.352351 (+ 3423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:43.357283 (+ 4932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:43.357289 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:43.357302 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:43.357827 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:43.358305 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:43.358308 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:43.358311 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:43.358587 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:43.359309 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:43.359320 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:43.359913 (+ 593us) client_negotiation.cc:769] Sending connection context
0810 00:57:43.360279 (+ 366us) client_negotiation.cc:240] Negotiation successful
0810 00:57:43.360966 (+ 687us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":381,"threads_started":1}
I20260810 00:57:43.384738 23064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.369128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:51779 (local address 127.20.81.190:38411)
0810 00:57:43.369337 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:43.369345 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:43.370667 (+ 1322us) server_negotiation.cc:407] Connection header received
0810 00:57:43.370926 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:43.370931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:43.371003 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:43.371107 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:43.372114 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.372983 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.374306 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.374784 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.374943 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.378042 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:43.378048 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:43.378053 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:43.378091 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:43.381331 (+ 3240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:43.382216 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:43.382223 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:43.382227 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:43.382371 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:43.382878 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:43.382885 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:43.382888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:43.383296 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:43.383441 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:43.384119 (+ 678us) server_negotiation.cc:299] Negotiation successful
0810 00:57:43.384350 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 00:57:43.384874 23306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.369303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38411 (local address 127.20.81.131:51779)
0810 00:57:43.370558 (+ 1255us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:43.370584 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:43.370751 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:43.371163 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:43.371168 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:43.371680 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:43.371972 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.371981 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.373135 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.373140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.374157 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.374168 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.375092 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.375100 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.375465 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.376213 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:43.376245 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:43.377802 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:43.381489 (+ 3687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:43.381494 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:43.381498 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:43.382098 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:43.382562 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:43.382568 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:43.382572 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:43.382754 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:43.383524 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:43.383528 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:43.383806 (+ 278us) client_negotiation.cc:769] Sending connection context
0810 00:57:43.384028 (+ 222us) client_negotiation.cc:240] Negotiation successful
0810 00:57:43.384388 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1033,"thread_start_us":783,"threads_started":1}
I20260810 00:57:43.390549 23194 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:34777
I20260810 00:57:43.390646 23351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:34777 every 8 connection(s)
I20260810 00:57:43.391873 23194 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 00:57:43.393875 23352 heartbeater.cc:347] Connected to a master server at 127.20.81.190:38411
I20260810 00:57:43.394109 23352 heartbeater.cc:464] Registering TS with master...
I20260810 00:57:43.394790 23352 heartbeater.cc:511] Master 127.20.81.190:38411 requested a full tablet report, sending...
I20260810 00:57:43.396057 22793 ts_manager.cc:194] Registered new tserver with Master: 294011f87b014d1682732a8b7dc02f0c (127.20.81.131:34777)
I20260810 00:57:43.396865 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 23194
I20260810 00:57:43.397050 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 00:57:43.397186 22793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:44445
I20260810 00:57:43.397671 20806 external_mini_cluster.cc:1428] Setting key 148ae8807521be9285c570440ea13323
I20260810 00:57:43.400588 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 00:57:43 dist-test-slave-gp6n krb5kdc[22743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 00:57:43 dist-test-slave-gp6n krb5kdc[22743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:43.453824 23064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.439888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41096 (local address 127.20.81.190:38411)
0810 00:57:43.440073 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:43.440078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:43.440301 (+ 223us) server_negotiation.cc:407] Connection header received
0810 00:57:43.440474 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:43.440479 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:43.440545 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:43.440639 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:43.442038 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.442901 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.444123 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.444549 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.444646 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.448678 (+ 4032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:43.448684 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:43.448687 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:43.448725 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:43.451548 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:43.452176 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:43.452180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:43.452182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:43.452296 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:43.452652 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:43.452655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:43.452657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:43.452931 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:43.453130 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:43.453399 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 00:57:43.453578 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 00:57:43.457878 22793 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:57:43.464730 22793 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:57:43.488951 23368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.477611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:34777 (local address 127.0.0.1:48356)
0810 00:57:43.479804 (+ 2193us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:43.479822 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:43.479963 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:43.481179 (+ 1216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:43.481184 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:43.481212 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:57:43.481688 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.481703 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.483565 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.483573 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.486158 (+ 2585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.486172 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.487931 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.487938 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.488144 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.488196 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 00:57:43.488336 (+ 140us) client_negotiation.cc:240] Negotiation successful
0810 00:57:43.488528 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2057,"mutex_wait_us":27,"thread_start_us":1931,"threads_started":1}
I20260810 00:57:43.488930 23369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.478461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48356 (local address 127.20.81.131:34777)
0810 00:57:43.480600 (+ 2139us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:43.480606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:43.480630 (+ 24us) server_negotiation.cc:407] Connection header received
0810 00:57:43.480697 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:43.480702 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:43.480959 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:43.481899 (+ 940us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:57:43.482266 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.483437 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.486326 (+ 2889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.487740 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.487877 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.488116 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:43.488399 (+ 283us) server_negotiation.cc:299] Negotiation successful
0810 00:57:43.488571 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1997,"thread_start_us":1817,"threads_started":1}
I20260810 00:57:43.490017 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.477567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45968 (local address 127.20.81.129:39145)
0810 00:57:43.478950 (+ 1383us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:43.478974 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:43.479703 (+ 729us) server_negotiation.cc:407] Connection header received
0810 00:57:43.480131 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:43.480141 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:43.480391 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:43.480567 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:57:43.481708 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.483706 (+ 1998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.486169 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.487740 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.487877 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.488094 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:43.489575 (+ 1481us) server_negotiation.cc:299] Negotiation successful
0810 00:57:43.489733 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1171,"thread_start_us":994,"threads_started":1}
I20260810 00:57:43.490829 23366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.477360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:39145 (local address 127.0.0.1:45968)
0810 00:57:43.479535 (+ 2175us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:43.479583 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:43.479854 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:43.480892 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:43.480902 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:43.480942 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:57:43.481476 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.481493 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.483892 (+ 2399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.483897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.486009 (+ 2112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.486021 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.488260 (+ 2239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.488265 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.488543 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.488575 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 00:57:43.490274 (+ 1699us) client_negotiation.cc:240] Negotiation successful
0810 00:57:43.490434 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1941,"mutex_wait_us":79,"thread_start_us":1803,"threads_started":1}
I20260810 00:57:43.491402 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.480573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:41307 (local address 127.0.0.1:45280)
0810 00:57:43.482325 (+ 1752us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:43.482346 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:43.482484 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:43.484771 (+ 2287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:43.484776 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:43.484799 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:57:43.485284 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.485296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.486859 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.486865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.488473 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.488485 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.490562 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.490568 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.490750 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.490789 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 00:57:43.490897 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 00:57:43.491047 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1643,"mutex_wait_us":1,"thread_start_us":1484,"threads_started":1}
I20260810 00:57:43.491575 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.480264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45280 (local address 127.20.81.130:41307)
0810 00:57:43.484024 (+ 3760us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:43.484032 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:43.484057 (+ 25us) server_negotiation.cc:407] Connection header received
0810 00:57:43.484156 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:43.484163 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:43.484362 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:43.484575 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:57:43.485473 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.486730 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.488647 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.490051 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.490194 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.490391 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:43.491085 (+ 694us) server_negotiation.cc:299] Negotiation successful
0810 00:57:43.491281 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3601,"thread_start_us":3471,"threads_started":1}
I20260810 00:57:43.493427 23267 tablet_service.cc:1467] Processing CreateTablet for tablet 5a84320340e243638347af78a7db58d3 (DEFAULT_TABLE table=test-table [id=20fcc341c21a44b5a51d812e40fe6fa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:57:43.494354 22950 tablet_service.cc:1467] Processing CreateTablet for tablet 5a84320340e243638347af78a7db58d3 (DEFAULT_TABLE table=test-table [id=20fcc341c21a44b5a51d812e40fe6fa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:57:43.495070 23267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a84320340e243638347af78a7db58d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:43.495067 23111 tablet_service.cc:1467] Processing CreateTablet for tablet 5a84320340e243638347af78a7db58d3 (DEFAULT_TABLE table=test-table [id=20fcc341c21a44b5a51d812e40fe6fa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:57:43.495751 22950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a84320340e243638347af78a7db58d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:43.496316 23111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a84320340e243638347af78a7db58d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:43.506260 23373 tablet_bootstrap.cc:492] T 5a84320340e243638347af78a7db58d3 P 294011f87b014d1682732a8b7dc02f0c: Bootstrap starting.
I20260810 00:57:43.511063 23373 tablet_bootstrap.cc:654] T 5a84320340e243638347af78a7db58d3 P 294011f87b014d1682732a8b7dc02f0c: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:43.511344 23374 tablet_bootstrap.cc:492] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f: Bootstrap starting.
I20260810 00:57:43.511924 23376 tablet_bootstrap.cc:492] T 5a84320340e243638347af78a7db58d3 P 492165fc756d4ec7a9b0d43f9624f6dc: Bootstrap starting.
I20260810 00:57:43.512470 23373 log.cc:824] T 5a84320340e243638347af78a7db58d3 P 294011f87b014d1682732a8b7dc02f0c: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:43.514937 23374 tablet_bootstrap.cc:654] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:43.515241 23376 tablet_bootstrap.cc:654] T 5a84320340e243638347af78a7db58d3 P 492165fc756d4ec7a9b0d43f9624f6dc: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:43.515717 23373 tablet_bootstrap.cc:492] T 5a84320340e243638347af78a7db58d3 P 294011f87b014d1682732a8b7dc02f0c: No bootstrap required, opened a new log
I20260810 00:57:43.516099 23374 log.cc:824] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:43.516455 23373 ts_tablet_manager.cc:1397] T 5a84320340e243638347af78a7db58d3 P 294011f87b014d1682732a8b7dc02f0c: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 00:57:43.516682 23376 log.cc:824] T 5a84320340e243638347af78a7db58d3 P 492165fc756d4ec7a9b0d43f9624f6dc: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:43.520015 23376 tablet_bootstrap.cc:492] T 5a84320340e243638347af78a7db58d3 P 492165fc756d4ec7a9b0d43f9624f6dc: No bootstrap required, opened a new log
I20260810 00:57:43.520272 23376 ts_tablet_manager.cc:1397] T 5a84320340e243638347af78a7db58d3 P 492165fc756d4ec7a9b0d43f9624f6dc: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.005s
I20260810 00:57:43.524453 23374 tablet_bootstrap.cc:492] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f: No bootstrap required, opened a new log
I20260810 00:57:43.524716 23374 ts_tablet_manager.cc:1397] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260810 00:57:43.527026 23373 raft_consensus.cc:348] T 5a84320340e243638347af78a7db58d3 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } }
I20260810 00:57:43.527477 23373 raft_consensus.cc:374] T 5a84320340e243638347af78a7db58d3 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:43.527585 23373 raft_consensus.cc:729] T 5a84320340e243638347af78a7db58d3 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 294011f87b014d1682732a8b7dc02f0c, State: Initialized, Role: FOLLOWER
I20260810 00:57:43.528221 23373 consensus_queue.cc:260] T 5a84320340e243638347af78a7db58d3 P 294011f87b014d1682732a8b7dc02f0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } }
I20260810 00:57:43.529345 23352 heartbeater.cc:503] Master 127.20.81.190:38411 was elected leader, sending a full tablet report...
I20260810 00:57:43.530045 23373 ts_tablet_manager.cc:1428] T 5a84320340e243638347af78a7db58d3 P 294011f87b014d1682732a8b7dc02f0c: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260810 00:57:43.530735 23376 raft_consensus.cc:348] T 5a84320340e243638347af78a7db58d3 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } }
I20260810 00:57:43.531165 23376 raft_consensus.cc:374] T 5a84320340e243638347af78a7db58d3 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:43.531267 23376 raft_consensus.cc:729] T 5a84320340e243638347af78a7db58d3 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 492165fc756d4ec7a9b0d43f9624f6dc, State: Initialized, Role: FOLLOWER
I20260810 00:57:43.531864 23376 consensus_queue.cc:260] T 5a84320340e243638347af78a7db58d3 P 492165fc756d4ec7a9b0d43f9624f6dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } }
I20260810 00:57:43.534732 23376 ts_tablet_manager.cc:1428] T 5a84320340e243638347af78a7db58d3 P 492165fc756d4ec7a9b0d43f9624f6dc: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260810 00:57:43.535506 23374 raft_consensus.cc:348] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } }
I20260810 00:57:43.535923 23374 raft_consensus.cc:374] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:43.536041 23374 raft_consensus.cc:729] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872da1bf65c242c6a9e85b515e34cb9f, State: Initialized, Role: FOLLOWER
I20260810 00:57:43.536662 23374 consensus_queue.cc:260] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } }
I20260810 00:57:43.537855 23190 heartbeater.cc:503] Master 127.20.81.190:38411 was elected leader, sending a full tablet report...
I20260810 00:57:43.538380 23374 ts_tablet_manager.cc:1428] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
W20260810 00:57:43.553846 23031 tablet.cc:2369] T 5a84320340e243638347af78a7db58d3 P 492165fc756d4ec7a9b0d43f9624f6dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:57:43.608036 23191 tablet.cc:2369] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:57:43.643743 23353 tablet.cc:2369] T 5a84320340e243638347af78a7db58d3 P 294011f87b014d1682732a8b7dc02f0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:57:43.713441 23382 raft_consensus.cc:482] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:57:43.713748 23382 raft_consensus.cc:504] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } }
I20260810 00:57:43.715564 23382 leader_election.cc:288] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 492165fc756d4ec7a9b0d43f9624f6dc (127.20.81.129:39145), 294011f87b014d1682732a8b7dc02f0c (127.20.81.131:34777)
I20260810 00:57:43.725395 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.715852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:52787 (local address 127.20.81.129:39145)
0810 00:57:43.716094 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:43.716102 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:43.717133 (+ 1031us) server_negotiation.cc:407] Connection header received
0810 00:57:43.717289 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:43.717294 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:43.717369 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:43.717457 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:57:43.718189 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.719300 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.722661 (+ 3361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.724044 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.724187 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.724267 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:43.725007 (+ 740us) server_negotiation.cc:299] Negotiation successful
0810 00:57:43.725140 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 00:57:43.725394 23369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.716320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:48063 (local address 127.20.81.131:34777)
0810 00:57:43.716526 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:43.716534 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:43.716841 (+ 307us) server_negotiation.cc:407] Connection header received
0810 00:57:43.717075 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:43.717080 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:43.717176 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:43.717288 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:57:43.718062 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.719247 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.722661 (+ 3414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.724045 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.724191 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.724267 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:43.725007 (+ 740us) server_negotiation.cc:299] Negotiation successful
0810 00:57:43.725140 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 00:57:43.725723 23386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.715911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:34777 (local address 127.20.81.130:48063)
0810 00:57:43.716727 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:43.716743 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:43.716926 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:43.717333 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:43.717344 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:43.717366 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:57:43.717894 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.717910 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.719437 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.719443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.722452 (+ 3009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.722469 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.724539 (+ 2070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.724544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.724820 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.724853 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 00:57:43.724963 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 00:57:43.725174 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"mutex_wait_us":320,"thread_start_us":685,"threads_started":1}
I20260810 00:57:43.725750 23384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.715948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:39145 (local address 127.20.81.130:52787)
0810 00:57:43.717027 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:43.717043 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:43.717158 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:43.717497 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:43.717501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:43.717543 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:57:43.718020 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.718034 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.719439 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.719443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.722452 (+ 3009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.722469 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.724533 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.724541 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.724812 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.724857 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 00:57:43.724963 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 00:57:43.725174 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":912,"mutex_wait_us":199,"thread_start_us":398,"threads_started":1}
I20260810 00:57:43.726871 22978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a84320340e243638347af78a7db58d3" candidate_uuid: "872da1bf65c242c6a9e85b515e34cb9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" is_pre_election: true
I20260810 00:57:43.726904 23292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a84320340e243638347af78a7db58d3" candidate_uuid: "872da1bf65c242c6a9e85b515e34cb9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "294011f87b014d1682732a8b7dc02f0c" is_pre_election: true
I20260810 00:57:43.727378 22978 raft_consensus.cc:2454] T 5a84320340e243638347af78a7db58d3 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 872da1bf65c242c6a9e85b515e34cb9f in term 0.
I20260810 00:57:43.727415 23292 raft_consensus.cc:2454] T 5a84320340e243638347af78a7db58d3 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 872da1bf65c242c6a9e85b515e34cb9f in term 0.
I20260810 00:57:43.728091 23058 leader_election.cc:302] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 492165fc756d4ec7a9b0d43f9624f6dc, 872da1bf65c242c6a9e85b515e34cb9f; no voters:
I20260810 00:57:43.728541 23382 raft_consensus.cc:2781] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:57:43.728709 23382 raft_consensus.cc:482] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:57:43.728839 23382 raft_consensus.cc:3037] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:43.731276 23382 raft_consensus.cc:504] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } }
I20260810 00:57:43.731994 23382 leader_election.cc:288] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f [CANDIDATE]: Term 1 election: Requested vote from peers 492165fc756d4ec7a9b0d43f9624f6dc (127.20.81.129:39145), 294011f87b014d1682732a8b7dc02f0c (127.20.81.131:34777)
I20260810 00:57:43.732370 22978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a84320340e243638347af78a7db58d3" candidate_uuid: "872da1bf65c242c6a9e85b515e34cb9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "492165fc756d4ec7a9b0d43f9624f6dc"
I20260810 00:57:43.732509 23292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a84320340e243638347af78a7db58d3" candidate_uuid: "872da1bf65c242c6a9e85b515e34cb9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "294011f87b014d1682732a8b7dc02f0c"
I20260810 00:57:43.732591 22978 raft_consensus.cc:3037] T 5a84320340e243638347af78a7db58d3 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:43.732744 23292 raft_consensus.cc:3037] T 5a84320340e243638347af78a7db58d3 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:43.735103 23292 raft_consensus.cc:2454] T 5a84320340e243638347af78a7db58d3 P 294011f87b014d1682732a8b7dc02f0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872da1bf65c242c6a9e85b515e34cb9f in term 1.
I20260810 00:57:43.735138 22978 raft_consensus.cc:2454] T 5a84320340e243638347af78a7db58d3 P 492165fc756d4ec7a9b0d43f9624f6dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872da1bf65c242c6a9e85b515e34cb9f in term 1.
I20260810 00:57:43.735538 23057 leader_election.cc:302] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 294011f87b014d1682732a8b7dc02f0c, 872da1bf65c242c6a9e85b515e34cb9f; no voters:
I20260810 00:57:43.735880 23382 raft_consensus.cc:2781] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:57:43.736375 23382 raft_consensus.cc:686] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f [term 1 LEADER]: Becoming Leader. State: Replica: 872da1bf65c242c6a9e85b515e34cb9f, State: Running, Role: LEADER
I20260810 00:57:43.736901 23382 consensus_queue.cc:237] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } }
I20260810 00:57:43.741886 22792 catalog_manager.cc:5495] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 872da1bf65c242c6a9e85b515e34cb9f (127.20.81.130). New cstate: current_term: 1 leader_uuid: "872da1bf65c242c6a9e85b515e34cb9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } health_report { overall_health: HEALTHY } } }
Aug 10 00:57:43 dist-test-slave-gp6n krb5kdc[22743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323461, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:43.815027 23395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.801202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38411 (local address 127.0.0.1:41102)
0810 00:57:43.801800 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:43.801815 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:43.801951 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:43.802336 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:43.802339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:43.803056 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:43.803316 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.803325 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.804346 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.804349 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.805416 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.805427 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.806195 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.806199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.806314 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.807104 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:43.807122 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:43.809579 (+ 2457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:43.812661 (+ 3082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:43.812670 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:43.812675 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:43.813076 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:43.813456 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:43.813459 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:43.813462 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:43.813585 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:43.814121 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:43.814131 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:43.814274 (+ 143us) client_negotiation.cc:769] Sending connection context
0810 00:57:43.814476 (+ 202us) client_negotiation.cc:240] Negotiation successful
0810 00:57:43.814700 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":347,"threads_started":1}
I20260810 00:57:43.815246 23064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.801031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41102 (local address 127.20.81.190:38411)
0810 00:57:43.801265 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:43.801273 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:43.801905 (+ 632us) server_negotiation.cc:407] Connection header received
0810 00:57:43.802060 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:43.802064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:43.802141 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:43.802237 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:43.803465 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.804233 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.805547 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.805978 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.806168 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.809779 (+ 3611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:43.809785 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:43.809788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:43.809818 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:43.812531 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:43.813183 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:43.813187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:43.813189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:43.813321 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:43.813708 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:43.813712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:43.813714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:43.814010 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:43.814153 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:43.814594 (+ 441us) server_negotiation.cc:299] Negotiation successful
0810 00:57:43.814864 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":63}
I20260810 00:57:43.820139 22792 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:41102:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 00:57:43.821121 22792 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:57:43.827977 23267 tablet_service.cc:1467] Processing CreateTablet for tablet ae52ecbfa2f74f0cb67c58537af2b3f4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e9e3329e0c724a8e83f3bc0c7af57fc6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 00:57:43.827877 22950 tablet_service.cc:1467] Processing CreateTablet for tablet ae52ecbfa2f74f0cb67c58537af2b3f4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e9e3329e0c724a8e83f3bc0c7af57fc6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 00:57:43.828001 23111 tablet_service.cc:1467] Processing CreateTablet for tablet ae52ecbfa2f74f0cb67c58537af2b3f4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e9e3329e0c724a8e83f3bc0c7af57fc6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 00:57:43.828886 22950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae52ecbfa2f74f0cb67c58537af2b3f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:43.828886 23267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae52ecbfa2f74f0cb67c58537af2b3f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:43.828979 23111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae52ecbfa2f74f0cb67c58537af2b3f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:43.835429 23374 tablet_bootstrap.cc:492] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 872da1bf65c242c6a9e85b515e34cb9f: Bootstrap starting.
I20260810 00:57:43.835983 23373 tablet_bootstrap.cc:492] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 294011f87b014d1682732a8b7dc02f0c: Bootstrap starting.
I20260810 00:57:43.836122 23376 tablet_bootstrap.cc:492] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc: Bootstrap starting.
I20260810 00:57:43.837357 23373 tablet_bootstrap.cc:654] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 294011f87b014d1682732a8b7dc02f0c: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:43.837356 23374 tablet_bootstrap.cc:654] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 872da1bf65c242c6a9e85b515e34cb9f: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:43.837960 23376 tablet_bootstrap.cc:654] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:43.839402 23373 tablet_bootstrap.cc:492] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 294011f87b014d1682732a8b7dc02f0c: No bootstrap required, opened a new log
I20260810 00:57:43.839632 23373 ts_tablet_manager.cc:1397] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 294011f87b014d1682732a8b7dc02f0c: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 00:57:43.840435 23376 tablet_bootstrap.cc:492] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc: No bootstrap required, opened a new log
I20260810 00:57:43.840592 23376 ts_tablet_manager.cc:1397] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 00:57:43.840478 23373 raft_consensus.cc:348] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:43.840778 23373 raft_consensus.cc:374] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:43.840883 23373 raft_consensus.cc:729] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 294011f87b014d1682732a8b7dc02f0c, State: Initialized, Role: FOLLOWER
I20260810 00:57:43.841078 23374 tablet_bootstrap.cc:492] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 872da1bf65c242c6a9e85b515e34cb9f: No bootstrap required, opened a new log
I20260810 00:57:43.841109 23373 consensus_queue.cc:260] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 294011f87b014d1682732a8b7dc02f0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:43.841252 23374 ts_tablet_manager.cc:1397] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 872da1bf65c242c6a9e85b515e34cb9f: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260810 00:57:43.841360 23376 raft_consensus.cc:348] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:43.841604 23376 raft_consensus.cc:374] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:43.841693 23376 raft_consensus.cc:729] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 492165fc756d4ec7a9b0d43f9624f6dc, State: Initialized, Role: FOLLOWER
I20260810 00:57:43.841579 23380 tablet_replica.cc:439] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 294011f87b014d1682732a8b7dc02f0c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } } }
I20260810 00:57:43.841946 23380 tablet_replica.cc:442] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 294011f87b014d1682732a8b7dc02f0c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:57:43.841910 23376 consensus_queue.cc:260] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:43.842065 23374 raft_consensus.cc:348] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:43.842365 23374 raft_consensus.cc:374] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:43.842445 23373 ts_tablet_manager.cc:1428] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 294011f87b014d1682732a8b7dc02f0c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 00:57:43.842489 23376 ts_tablet_manager.cc:1428] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 00:57:43.842567 23374 raft_consensus.cc:729] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872da1bf65c242c6a9e85b515e34cb9f, State: Initialized, Role: FOLLOWER
I20260810 00:57:43.842891 23381 tablet_replica.cc:439] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } } }
I20260810 00:57:43.843266 23381 tablet_replica.cc:442] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:57:43.843633 23374 consensus_queue.cc:260] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 872da1bf65c242c6a9e85b515e34cb9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:43.844383 23374 ts_tablet_manager.cc:1428] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 872da1bf65c242c6a9e85b515e34cb9f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 00:57:43.844215 23382 tablet_replica.cc:439] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 872da1bf65c242c6a9e85b515e34cb9f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } } }
I20260810 00:57:43.845186 23382 tablet_replica.cc:442] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 872da1bf65c242c6a9e85b515e34cb9f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:57:43.846143 23381 raft_consensus.cc:482] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:57:43.846328 23381 raft_consensus.cc:504] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:43.848133 23381 leader_election.cc:288] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 872da1bf65c242c6a9e85b515e34cb9f (127.20.81.130:41307), 294011f87b014d1682732a8b7dc02f0c (127.20.81.131:34777)
I20260810 00:57:43.856325 23369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.848685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:58223 (local address 127.20.81.131:34777)
0810 00:57:43.848842 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:43.848847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:43.849453 (+ 606us) server_negotiation.cc:407] Connection header received
0810 00:57:43.849753 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:43.849758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:43.849828 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:43.849921 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:57:43.850573 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.851611 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.853967 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.855172 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.855280 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.855347 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:43.855919 (+ 572us) server_negotiation.cc:299] Negotiation successful
0810 00:57:43.856059 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 00:57:43.856488 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.849488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:40015 (local address 127.20.81.130:41307)
0810 00:57:43.849664 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:43.849672 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:43.850127 (+ 455us) server_negotiation.cc:407] Connection header received
0810 00:57:43.850293 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:43.850300 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:43.850371 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:43.850490 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:57:43.851025 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.852036 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.854361 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.855368 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.855566 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.855661 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:43.855987 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 00:57:43.856140 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 00:57:43.856504 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.849343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:41307 (local address 127.20.81.129:40015)
0810 00:57:43.850036 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:43.850056 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:43.850176 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:43.850530 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:43.850533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:43.850553 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:57:43.850832 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.850863 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.852188 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.852193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.854108 (+ 1915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.854121 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.855640 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.855647 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.855810 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.855842 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 00:57:43.855957 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 00:57:43.856091 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":4,"thread_start_us":357,"threads_started":1}
I20260810 00:57:43.857366 23399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.848686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:34777 (local address 127.20.81.129:58223)
0810 00:57:43.849346 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:43.849368 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:43.849568 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:43.849957 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:43.849961 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:43.849985 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:57:43.850361 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.850373 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.851780 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.851785 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.853772 (+ 1987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.853784 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.855539 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.855545 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.855720 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.855758 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 00:57:43.856880 (+ 1122us) client_negotiation.cc:240] Negotiation successful
0810 00:57:43.857012 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"mutex_wait_us":19,"thread_start_us":792,"threads_started":1}
I20260810 00:57:43.857337 23140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae52ecbfa2f74f0cb67c58537af2b3f4" candidate_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872da1bf65c242c6a9e85b515e34cb9f" is_pre_election: true
I20260810 00:57:43.857774 23140 raft_consensus.cc:2454] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 492165fc756d4ec7a9b0d43f9624f6dc in term 0.
I20260810 00:57:43.858189 23292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae52ecbfa2f74f0cb67c58537af2b3f4" candidate_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "294011f87b014d1682732a8b7dc02f0c" is_pre_election: true
I20260810 00:57:43.858464 23292 raft_consensus.cc:2454] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 492165fc756d4ec7a9b0d43f9624f6dc in term 0.
I20260810 00:57:43.858634 22895 leader_election.cc:302] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 492165fc756d4ec7a9b0d43f9624f6dc, 872da1bf65c242c6a9e85b515e34cb9f; no voters:
I20260810 00:57:43.859216 23381 raft_consensus.cc:2781] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:57:43.859393 23381 raft_consensus.cc:482] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:57:43.859509 23381 raft_consensus.cc:3037] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:43.861763 23381 raft_consensus.cc:504] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:43.862485 23381 leader_election.cc:288] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc [CANDIDATE]: Term 1 election: Requested vote from peers 872da1bf65c242c6a9e85b515e34cb9f (127.20.81.130:41307), 294011f87b014d1682732a8b7dc02f0c (127.20.81.131:34777)
I20260810 00:57:43.862912 23292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae52ecbfa2f74f0cb67c58537af2b3f4" candidate_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "294011f87b014d1682732a8b7dc02f0c"
I20260810 00:57:43.863142 23292 raft_consensus.cc:3037] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:43.863155 23140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae52ecbfa2f74f0cb67c58537af2b3f4" candidate_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872da1bf65c242c6a9e85b515e34cb9f"
I20260810 00:57:43.863461 23140 raft_consensus.cc:3037] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:43.866182 23292 raft_consensus.cc:2454] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 294011f87b014d1682732a8b7dc02f0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 492165fc756d4ec7a9b0d43f9624f6dc in term 1.
I20260810 00:57:43.866452 23140 raft_consensus.cc:2454] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 872da1bf65c242c6a9e85b515e34cb9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 492165fc756d4ec7a9b0d43f9624f6dc in term 1.
I20260810 00:57:43.866719 22894 leader_election.cc:302] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 294011f87b014d1682732a8b7dc02f0c, 492165fc756d4ec7a9b0d43f9624f6dc; no voters:
I20260810 00:57:43.867450 23381 raft_consensus.cc:2781] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:57:43.867966 23381 raft_consensus.cc:686] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc [term 1 LEADER]: Becoming Leader. State: Replica: 492165fc756d4ec7a9b0d43f9624f6dc, State: Running, Role: LEADER
I20260810 00:57:43.868778 23381 consensus_queue.cc:237] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:43.873258 23402 tablet_replica.cc:439] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc: TxnStatusTablet state changed. Reason: New leader 492165fc756d4ec7a9b0d43f9624f6dc. Latest consensus state: current_term: 1 leader_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } } }
I20260810 00:57:43.873603 23402 tablet_replica.cc:442] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:57:43.874367 23404 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 00:57:43.874713 23404 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 00:57:43.874198 22791 catalog_manager.cc:5495] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 492165fc756d4ec7a9b0d43f9624f6dc (127.20.81.129). New cstate: current_term: 1 leader_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } health_report { overall_health: UNKNOWN } } }
I20260810 00:57:43.902947 23395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.895711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:39145 (local address 127.0.0.1:45974)
0810 00:57:43.895952 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:43.895974 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:43.896088 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:43.896634 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:43.896637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:43.896647 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:57:43.896805 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.896812 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.898352 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.898356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.899888 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.899924 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.900989 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.900993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.901216 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.901271 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:57:43.902375 (+ 1104us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:57:43.902396 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 00:57:43.902576 (+ 180us) client_negotiation.cc:240] Negotiation successful
0810 00:57:43.902696 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 00:57:43.903095 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.895748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45974 (local address 127.20.81.129:39145)
0810 00:57:43.895959 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:43.895966 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:43.896040 (+ 74us) server_negotiation.cc:407] Connection header received
0810 00:57:43.896282 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:43.896289 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:43.896430 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:43.896612 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:43.896935 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.898229 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.900075 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.900814 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.900988 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.901496 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:43.902230 (+ 734us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:43.902449 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:43.902630 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 00:57:43.902727 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 00:57:43.911216 23140 raft_consensus.cc:1261] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 872da1bf65c242c6a9e85b515e34cb9f [term 1 FOLLOWER]: Refusing update from remote peer 492165fc756d4ec7a9b0d43f9624f6dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:57:43.911262 23292 raft_consensus.cc:1261] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 294011f87b014d1682732a8b7dc02f0c [term 1 FOLLOWER]: Refusing update from remote peer 492165fc756d4ec7a9b0d43f9624f6dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:57:43.912186 23402 consensus_queue.cc:1035] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:57:43.912503 23381 consensus_queue.cc:1035] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:57:43.925699 23382 tablet_replica.cc:439] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 872da1bf65c242c6a9e85b515e34cb9f: TxnStatusTablet state changed. Reason: New leader 492165fc756d4ec7a9b0d43f9624f6dc. Latest consensus state: current_term: 1 leader_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } } }
I20260810 00:57:43.926049 23382 tablet_replica.cc:442] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 872da1bf65c242c6a9e85b515e34cb9f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:57:43.934800 23402 tablet_replica.cc:439] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } } }
I20260810 00:57:43.935241 23402 tablet_replica.cc:442] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:57:43.936614 23380 tablet_replica.cc:439] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 294011f87b014d1682732a8b7dc02f0c: TxnStatusTablet state changed. Reason: New leader 492165fc756d4ec7a9b0d43f9624f6dc. Latest consensus state: current_term: 1 leader_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } } }
I20260810 00:57:43.936887 23380 tablet_replica.cc:442] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 294011f87b014d1682732a8b7dc02f0c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:57:43.942503 23402 tablet_replica.cc:439] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } } }
I20260810 00:57:43.942878 23402 tablet_replica.cc:442] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 492165fc756d4ec7a9b0d43f9624f6dc: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:57:43.945981 23410 mvcc.cc:205] Tried to move back new op lower bound from 7316780908169891840 to 7316780908011126784. Current Snapshot: MvccSnapshot[applied={T|T < 7316780908169891840}]
I20260810 00:57:43.947261 23382 tablet_replica.cc:439] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 872da1bf65c242c6a9e85b515e34cb9f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } } }
I20260810 00:57:43.947577 23382 tablet_replica.cc:442] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 872da1bf65c242c6a9e85b515e34cb9f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:57:43.949692 23380 tablet_replica.cc:439] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 294011f87b014d1682732a8b7dc02f0c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } } }
I20260810 00:57:43.949927 23380 tablet_replica.cc:442] T ae52ecbfa2f74f0cb67c58537af2b3f4 P 294011f87b014d1682732a8b7dc02f0c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:57:43.963649 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.956987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45288 (local address 127.20.81.130:41307)
0810 00:57:43.957176 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:43.957182 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:43.957601 (+ 419us) server_negotiation.cc:407] Connection header received
0810 00:57:43.957764 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:43.957770 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:43.957876 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:43.958033 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:43.958433 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.959585 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.960931 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.961578 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.961714 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.962087 (+ 373us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:43.962855 (+ 768us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:43.963037 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:43.963223 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 00:57:43.963317 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 00:57:43.981724 23422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.975268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:39145 (local address 127.20.81.130:54899)
0810 00:57:43.976045 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:43.976102 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:43.976235 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:43.976607 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:43.976610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:43.976620 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:57:43.976902 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.976910 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.978470 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.978479 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.979633 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:43.979645 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.980663 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.980667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:43.980780 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.980833 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:57:43.981263 (+ 430us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:57:43.981272 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 00:57:43.981330 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 00:57:43.981434 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":429,"threads_started":1}
I20260810 00:57:43.981792 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:43.975265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:54899 (local address 127.20.81.129:39145)
0810 00:57:43.975480 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:43.975486 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:43.976195 (+ 709us) server_negotiation.cc:407] Connection header received
0810 00:57:43.976392 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:43.976398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:43.976492 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:43.976599 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:43.977051 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.978248 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.979783 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:43.980493 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:43.980618 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:43.981030 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:43.981166 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:43.981259 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:43.981386 (+ 127us) server_negotiation.cc:299] Negotiation successful
0810 00:57:43.981468 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 00:57:43.991973 23291 raft_consensus.cc:1261] T 5a84320340e243638347af78a7db58d3 P 294011f87b014d1682732a8b7dc02f0c [term 1 FOLLOWER]: Refusing update from remote peer 872da1bf65c242c6a9e85b515e34cb9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:57:43.992404 22978 raft_consensus.cc:1261] T 5a84320340e243638347af78a7db58d3 P 492165fc756d4ec7a9b0d43f9624f6dc [term 1 FOLLOWER]: Refusing update from remote peer 872da1bf65c242c6a9e85b515e34cb9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:57:43.992938 23387 consensus_queue.cc:1035] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:57:43.993284 23382 consensus_queue.cc:1035] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:57:43.998965 23409 mvcc.cc:205] Tried to move back new op lower bound from 7316780908505096192 to 7316780907470663680. Current Snapshot: MvccSnapshot[applied={T|T < 7316780908505096192}]
I20260810 00:57:44.045382 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:44.034318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:38221 (local address 127.20.81.130:41307)
0810 00:57:44.034519 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:44.034525 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:44.039480 (+ 4955us) server_negotiation.cc:407] Connection header received
0810 00:57:44.039734 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:44.039738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:44.039808 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:44.039885 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:44.040554 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:44.041281 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:44.043511 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:44.044141 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:44.044284 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:44.044792 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:44.044894 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:44.044983 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:44.045109 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 00:57:44.045174 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 00:57:44.045704 23431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:44.034341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:41307 (local address 127.20.81.129:38221)
0810 00:57:44.039371 (+ 5030us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:44.039395 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:44.039599 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:44.039934 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:44.039940 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:44.039953 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:57:44.040340 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:44.040358 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:44.041421 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:44.041427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:44.043351 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:44.043367 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:44.044332 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:44.044339 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:44.044529 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:44.044606 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:57:44.044999 (+ 393us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:57:44.045010 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 00:57:44.045098 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 00:57:44.045255 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4407,"thread_start_us":4245,"threads_started":1}
I20260810 00:57:44.084208 22792 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:57:44.084524 22792 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:57:44.089027 22792 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0: Sending DeleteTablet for 3 replicas of tablet 5a84320340e243638347af78a7db58d3
I20260810 00:57:44.090349 23267 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a84320340e243638347af78a7db58d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:57:44 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:48356
I20260810 00:57:44.090349 22950 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a84320340e243638347af78a7db58d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:57:44 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:45968
I20260810 00:57:44.091984 23111 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a84320340e243638347af78a7db58d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:57:44 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:45280
I20260810 00:57:44.093914 22791 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:57:44.094796 22791 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:57:44.103281 23437 tablet_replica.cc:330] T 5a84320340e243638347af78a7db58d3 P 492165fc756d4ec7a9b0d43f9624f6dc: stopping tablet replica
I20260810 00:57:44.103829 23437 raft_consensus.cc:2229] T 5a84320340e243638347af78a7db58d3 P 492165fc756d4ec7a9b0d43f9624f6dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:57:44.104194 23437 raft_consensus.cc:2258] T 5a84320340e243638347af78a7db58d3 P 492165fc756d4ec7a9b0d43f9624f6dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:57:44.104589 23266 tablet_service.cc:1467] Processing CreateTablet for tablet ae057fd3d8224beb9b03292a9358a288 (DEFAULT_TABLE table=test-table [id=135d50101a8d46dbb730bdc9e3e886c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:57:44.105166 23266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae057fd3d8224beb9b03292a9358a288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:44.105432 23109 tablet_service.cc:1467] Processing CreateTablet for tablet ae057fd3d8224beb9b03292a9358a288 (DEFAULT_TABLE table=test-table [id=135d50101a8d46dbb730bdc9e3e886c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:57:44.106166 23109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae057fd3d8224beb9b03292a9358a288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:44.110179 22950 tablet_service.cc:1467] Processing CreateTablet for tablet ae057fd3d8224beb9b03292a9358a288 (DEFAULT_TABLE table=test-table [id=135d50101a8d46dbb730bdc9e3e886c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:57:44.110698 22950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae057fd3d8224beb9b03292a9358a288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:44.111033 23438 tablet_replica.cc:330] T 5a84320340e243638347af78a7db58d3 P 294011f87b014d1682732a8b7dc02f0c: stopping tablet replica
I20260810 00:57:44.112522 23438 raft_consensus.cc:2229] T 5a84320340e243638347af78a7db58d3 P 294011f87b014d1682732a8b7dc02f0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:57:44.112926 23373 tablet_bootstrap.cc:492] T ae057fd3d8224beb9b03292a9358a288 P 294011f87b014d1682732a8b7dc02f0c: Bootstrap starting.
I20260810 00:57:44.113049 23438 raft_consensus.cc:2258] T 5a84320340e243638347af78a7db58d3 P 294011f87b014d1682732a8b7dc02f0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:57:44.114778 23373 tablet_bootstrap.cc:654] T ae057fd3d8224beb9b03292a9358a288 P 294011f87b014d1682732a8b7dc02f0c: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:44.116803 23438 ts_tablet_manager.cc:1905] T 5a84320340e243638347af78a7db58d3 P 294011f87b014d1682732a8b7dc02f0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:57:44.121272 23440 tablet_replica.cc:330] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f: stopping tablet replica
I20260810 00:57:44.121726 23440 raft_consensus.cc:2229] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:57:44.121918 23437 ts_tablet_manager.cc:1905] T 5a84320340e243638347af78a7db58d3 P 492165fc756d4ec7a9b0d43f9624f6dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:57:44.122430 23440 raft_consensus.cc:2258] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:57:44.122634 23374 tablet_bootstrap.cc:492] T ae057fd3d8224beb9b03292a9358a288 P 872da1bf65c242c6a9e85b515e34cb9f: Bootstrap starting.
I20260810 00:57:44.124529 23374 tablet_bootstrap.cc:654] T ae057fd3d8224beb9b03292a9358a288 P 872da1bf65c242c6a9e85b515e34cb9f: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:44.128515 23438 ts_tablet_manager.cc:1918] T 5a84320340e243638347af78a7db58d3 P 294011f87b014d1682732a8b7dc02f0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:57:44.128743 23438 log.cc:1196] T 5a84320340e243638347af78a7db58d3 P 294011f87b014d1682732a8b7dc02f0c: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/5a84320340e243638347af78a7db58d3
I20260810 00:57:44.129391 23438 ts_tablet_manager.cc:1939] T 5a84320340e243638347af78a7db58d3 P 294011f87b014d1682732a8b7dc02f0c: Deleting consensus metadata
I20260810 00:57:44.131606 22778 catalog_manager.cc:4841] TS 294011f87b014d1682732a8b7dc02f0c (127.20.81.131:34777): tablet 5a84320340e243638347af78a7db58d3 (table test-table [id=20fcc341c21a44b5a51d812e40fe6fa3]) successfully deleted
I20260810 00:57:44.133173 23373 tablet_bootstrap.cc:492] T ae057fd3d8224beb9b03292a9358a288 P 294011f87b014d1682732a8b7dc02f0c: No bootstrap required, opened a new log
I20260810 00:57:44.134691 23373 ts_tablet_manager.cc:1397] T ae057fd3d8224beb9b03292a9358a288 P 294011f87b014d1682732a8b7dc02f0c: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.009s
I20260810 00:57:44.135591 23373 raft_consensus.cc:348] T ae057fd3d8224beb9b03292a9358a288 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:44.136013 23373 raft_consensus.cc:374] T ae057fd3d8224beb9b03292a9358a288 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:44.136137 23373 raft_consensus.cc:729] T ae057fd3d8224beb9b03292a9358a288 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 294011f87b014d1682732a8b7dc02f0c, State: Initialized, Role: FOLLOWER
I20260810 00:57:44.136204 23440 ts_tablet_manager.cc:1905] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:57:44.136530 23373 consensus_queue.cc:260] T ae057fd3d8224beb9b03292a9358a288 P 294011f87b014d1682732a8b7dc02f0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:44.137567 23373 ts_tablet_manager.cc:1428] T ae057fd3d8224beb9b03292a9358a288 P 294011f87b014d1682732a8b7dc02f0c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260810 00:57:44.138288 23376 tablet_bootstrap.cc:492] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc: Bootstrap starting.
I20260810 00:57:44.140175 23376 tablet_bootstrap.cc:654] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:44.147461 23440 ts_tablet_manager.cc:1918] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:57:44.147670 23440 log.cc:1196] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/5a84320340e243638347af78a7db58d3
I20260810 00:57:44.148293 23440 ts_tablet_manager.cc:1939] T 5a84320340e243638347af78a7db58d3 P 872da1bf65c242c6a9e85b515e34cb9f: Deleting consensus metadata
I20260810 00:57:44.149400 23376 tablet_bootstrap.cc:492] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc: No bootstrap required, opened a new log
I20260810 00:57:44.149597 23376 ts_tablet_manager.cc:1397] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.002s
I20260810 00:57:44.150209 22779 catalog_manager.cc:4841] TS 872da1bf65c242c6a9e85b515e34cb9f (127.20.81.130:41307): tablet 5a84320340e243638347af78a7db58d3 (table test-table [id=20fcc341c21a44b5a51d812e40fe6fa3]) successfully deleted
I20260810 00:57:44.150485 23376 raft_consensus.cc:348] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:44.150789 23376 raft_consensus.cc:374] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:44.150892 23376 raft_consensus.cc:729] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 492165fc756d4ec7a9b0d43f9624f6dc, State: Initialized, Role: FOLLOWER
I20260810 00:57:44.151201 23376 consensus_queue.cc:260] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:44.154345 23376 ts_tablet_manager.cc:1428] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260810 00:57:44.155820 23374 tablet_bootstrap.cc:492] T ae057fd3d8224beb9b03292a9358a288 P 872da1bf65c242c6a9e85b515e34cb9f: No bootstrap required, opened a new log
I20260810 00:57:44.156029 23374 ts_tablet_manager.cc:1397] T ae057fd3d8224beb9b03292a9358a288 P 872da1bf65c242c6a9e85b515e34cb9f: Time spent bootstrapping tablet: real 0.033s user 0.007s sys 0.005s
I20260810 00:57:44.156870 23374 raft_consensus.cc:348] T ae057fd3d8224beb9b03292a9358a288 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:44.157135 23374 raft_consensus.cc:374] T ae057fd3d8224beb9b03292a9358a288 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:44.157227 23374 raft_consensus.cc:729] T ae057fd3d8224beb9b03292a9358a288 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872da1bf65c242c6a9e85b515e34cb9f, State: Initialized, Role: FOLLOWER
I20260810 00:57:44.157492 23374 consensus_queue.cc:260] T ae057fd3d8224beb9b03292a9358a288 P 872da1bf65c242c6a9e85b515e34cb9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:44.158354 23437 ts_tablet_manager.cc:1918] T 5a84320340e243638347af78a7db58d3 P 492165fc756d4ec7a9b0d43f9624f6dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:57:44.158577 23437 log.cc:1196] T 5a84320340e243638347af78a7db58d3 P 492165fc756d4ec7a9b0d43f9624f6dc: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-0/wal/wals/5a84320340e243638347af78a7db58d3
I20260810 00:57:44.159209 23437 ts_tablet_manager.cc:1939] T 5a84320340e243638347af78a7db58d3 P 492165fc756d4ec7a9b0d43f9624f6dc: Deleting consensus metadata
I20260810 00:57:44.160399 23374 ts_tablet_manager.cc:1428] T ae057fd3d8224beb9b03292a9358a288 P 872da1bf65c242c6a9e85b515e34cb9f: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260810 00:57:44.161053 22779 catalog_manager.cc:4841] TS 492165fc756d4ec7a9b0d43f9624f6dc (127.20.81.129:39145): tablet 5a84320340e243638347af78a7db58d3 (table test-table [id=20fcc341c21a44b5a51d812e40fe6fa3]) successfully deleted
I20260810 00:57:44.318176 23424 raft_consensus.cc:482] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:57:44.318722 23424 raft_consensus.cc:504] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:44.319882 23424 leader_election.cc:288] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 872da1bf65c242c6a9e85b515e34cb9f (127.20.81.130:41307), 294011f87b014d1682732a8b7dc02f0c (127.20.81.131:34777)
I20260810 00:57:44.320779 23140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae057fd3d8224beb9b03292a9358a288" candidate_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872da1bf65c242c6a9e85b515e34cb9f" is_pre_election: true
I20260810 00:57:44.321091 23140 raft_consensus.cc:2454] T ae057fd3d8224beb9b03292a9358a288 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 492165fc756d4ec7a9b0d43f9624f6dc in term 0.
I20260810 00:57:44.322141 23290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae057fd3d8224beb9b03292a9358a288" candidate_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "294011f87b014d1682732a8b7dc02f0c" is_pre_election: true
I20260810 00:57:44.322474 23290 raft_consensus.cc:2454] T ae057fd3d8224beb9b03292a9358a288 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 492165fc756d4ec7a9b0d43f9624f6dc in term 0.
I20260810 00:57:44.323143 22895 leader_election.cc:302] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 492165fc756d4ec7a9b0d43f9624f6dc, 872da1bf65c242c6a9e85b515e34cb9f; no voters:
I20260810 00:57:44.323962 23424 raft_consensus.cc:2781] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:57:44.324096 23424 raft_consensus.cc:482] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:57:44.324194 23424 raft_consensus.cc:3037] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:44.328259 23424 raft_consensus.cc:504] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:44.329669 23140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae057fd3d8224beb9b03292a9358a288" candidate_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872da1bf65c242c6a9e85b515e34cb9f"
I20260810 00:57:44.329932 23140 raft_consensus.cc:3037] T ae057fd3d8224beb9b03292a9358a288 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:44.334937 23140 raft_consensus.cc:2454] T ae057fd3d8224beb9b03292a9358a288 P 872da1bf65c242c6a9e85b515e34cb9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 492165fc756d4ec7a9b0d43f9624f6dc in term 1.
I20260810 00:57:44.335551 22895 leader_election.cc:302] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 492165fc756d4ec7a9b0d43f9624f6dc, 872da1bf65c242c6a9e85b515e34cb9f; no voters:
I20260810 00:57:44.336004 23381 raft_consensus.cc:2781] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:57:44.336408 23424 leader_election.cc:288] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc [CANDIDATE]: Term 1 election: Requested vote from peers 872da1bf65c242c6a9e85b515e34cb9f (127.20.81.130:41307), 294011f87b014d1682732a8b7dc02f0c (127.20.81.131:34777)
I20260810 00:57:44.336572 23381 raft_consensus.cc:686] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc [term 1 LEADER]: Becoming Leader. State: Replica: 492165fc756d4ec7a9b0d43f9624f6dc, State: Running, Role: LEADER
I20260810 00:57:44.337121 23381 consensus_queue.cc:237] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:44.337387 23290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae057fd3d8224beb9b03292a9358a288" candidate_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "294011f87b014d1682732a8b7dc02f0c"
I20260810 00:57:44.337610 23290 raft_consensus.cc:3037] T ae057fd3d8224beb9b03292a9358a288 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:44.341899 23290 raft_consensus.cc:2454] T ae057fd3d8224beb9b03292a9358a288 P 294011f87b014d1682732a8b7dc02f0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 492165fc756d4ec7a9b0d43f9624f6dc in term 1.
I20260810 00:57:44.344971 22792 catalog_manager.cc:5495] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 492165fc756d4ec7a9b0d43f9624f6dc (127.20.81.129). New cstate: current_term: 1 leader_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } health_report { overall_health: UNKNOWN } } }
I20260810 00:57:44.427546 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:44.422604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45978 (local address 127.20.81.129:39145)
0810 00:57:44.422775 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:44.422780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:44.422882 (+ 102us) server_negotiation.cc:407] Connection header received
0810 00:57:44.423072 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:44.423076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:44.423143 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:44.423244 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:44.423620 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:44.424399 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:44.425709 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:44.426179 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:44.426313 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:44.426773 (+ 460us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:44.426913 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:44.427015 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:44.427158 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 00:57:44.427234 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 00:57:44.441026 23431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:44.435351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:39145 (local address 127.20.81.129:34139)
0810 00:57:44.435570 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:44.435599 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:44.435740 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:44.436300 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:44.436305 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:44.436317 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:57:44.436463 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:44.436470 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:44.437672 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:44.437676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:44.438852 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:44.438864 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:44.439666 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:44.439673 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:44.439840 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:44.439874 (+ 34us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:57:44.440344 (+ 470us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:57:44.440357 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 00:57:44.440485 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 00:57:44.440649 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 00:57:44.441026 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:44.435753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:34139 (local address 127.20.81.129:39145)
0810 00:57:44.435943 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:44.435947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:44.435960 (+ 13us) server_negotiation.cc:407] Connection header received
0810 00:57:44.436003 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:44.436007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:44.436062 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:44.436180 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:44.436651 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:44.437507 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:44.438997 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:44.439463 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:44.439633 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:44.440058 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:44.440202 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:44.440332 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:44.440507 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 00:57:44.440594 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 00:57:44.453297 23290 raft_consensus.cc:1261] T ae057fd3d8224beb9b03292a9358a288 P 294011f87b014d1682732a8b7dc02f0c [term 1 FOLLOWER]: Refusing update from remote peer 492165fc756d4ec7a9b0d43f9624f6dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:57:44.453320 23140 raft_consensus.cc:1261] T ae057fd3d8224beb9b03292a9358a288 P 872da1bf65c242c6a9e85b515e34cb9f [term 1 FOLLOWER]: Refusing update from remote peer 492165fc756d4ec7a9b0d43f9624f6dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:57:44.454093 23402 consensus_queue.cc:1035] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:57:44.454516 23424 consensus_queue.cc:1035] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:57:44.467816 23408 mvcc.cc:205] Tried to move back new op lower bound from 7316780910395351040 to 7316780909928030208. Current Snapshot: MvccSnapshot[applied={T|T < 7316780910395351040}]
W20260810 00:57:44.493420 23408 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 00:57:46.091020 22793 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:57:46.091354 22793 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:57:46.095307 22793 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0: Sending DeleteTablet for 3 replicas of tablet ae057fd3d8224beb9b03292a9358a288
I20260810 00:57:46.096163 22949 tablet_service.cc:1514] Processing DeleteTablet for tablet ae057fd3d8224beb9b03292a9358a288 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:57:46 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:45968
I20260810 00:57:46.096315 23109 tablet_service.cc:1514] Processing DeleteTablet for tablet ae057fd3d8224beb9b03292a9358a288 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:57:46 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:45280
I20260810 00:57:46.096323 23266 tablet_service.cc:1514] Processing DeleteTablet for tablet ae057fd3d8224beb9b03292a9358a288 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:57:46 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:48356
I20260810 00:57:46.108860 23475 tablet_replica.cc:330] T ae057fd3d8224beb9b03292a9358a288 P 294011f87b014d1682732a8b7dc02f0c: stopping tablet replica
I20260810 00:57:46.109308 23475 raft_consensus.cc:2229] T ae057fd3d8224beb9b03292a9358a288 P 294011f87b014d1682732a8b7dc02f0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:57:46.109447 23474 tablet_replica.cc:330] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc: stopping tablet replica
I20260810 00:57:46.109678 23475 raft_consensus.cc:2258] T ae057fd3d8224beb9b03292a9358a288 P 294011f87b014d1682732a8b7dc02f0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:57:46.109804 23474 raft_consensus.cc:2229] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:57:46.111016 23474 raft_consensus.cc:2258] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:57:46.111124 23475 ts_tablet_manager.cc:1905] T ae057fd3d8224beb9b03292a9358a288 P 294011f87b014d1682732a8b7dc02f0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:57:46.111795 23476 tablet_replica.cc:330] T ae057fd3d8224beb9b03292a9358a288 P 872da1bf65c242c6a9e85b515e34cb9f: stopping tablet replica
I20260810 00:57:46.112035 23476 raft_consensus.cc:2229] T ae057fd3d8224beb9b03292a9358a288 P 872da1bf65c242c6a9e85b515e34cb9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:57:46.112313 23476 raft_consensus.cc:2258] T ae057fd3d8224beb9b03292a9358a288 P 872da1bf65c242c6a9e85b515e34cb9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:57:46.112602 23474 ts_tablet_manager.cc:1905] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:57:46.115052 23476 ts_tablet_manager.cc:1905] T ae057fd3d8224beb9b03292a9358a288 P 872da1bf65c242c6a9e85b515e34cb9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:57:46.117435 23475 ts_tablet_manager.cc:1918] T ae057fd3d8224beb9b03292a9358a288 P 294011f87b014d1682732a8b7dc02f0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 00:57:46.117630 23475 log.cc:1196] T ae057fd3d8224beb9b03292a9358a288 P 294011f87b014d1682732a8b7dc02f0c: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/ae057fd3d8224beb9b03292a9358a288
I20260810 00:57:46.118153 23475 ts_tablet_manager.cc:1939] T ae057fd3d8224beb9b03292a9358a288 P 294011f87b014d1682732a8b7dc02f0c: Deleting consensus metadata
I20260810 00:57:46.119400 22778 catalog_manager.cc:4841] TS 294011f87b014d1682732a8b7dc02f0c (127.20.81.131:34777): tablet ae057fd3d8224beb9b03292a9358a288 (table test-table [id=135d50101a8d46dbb730bdc9e3e886c8]) successfully deleted
I20260810 00:57:46.120329 23474 ts_tablet_manager.cc:1918] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 00:57:46.120476 23474 log.cc:1196] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-0/wal/wals/ae057fd3d8224beb9b03292a9358a288
I20260810 00:57:46.120952 23474 ts_tablet_manager.cc:1939] T ae057fd3d8224beb9b03292a9358a288 P 492165fc756d4ec7a9b0d43f9624f6dc: Deleting consensus metadata
I20260810 00:57:46.121479 23476 ts_tablet_manager.cc:1918] T ae057fd3d8224beb9b03292a9358a288 P 872da1bf65c242c6a9e85b515e34cb9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 00:57:46.121631 23476 log.cc:1196] T ae057fd3d8224beb9b03292a9358a288 P 872da1bf65c242c6a9e85b515e34cb9f: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/ae057fd3d8224beb9b03292a9358a288
I20260810 00:57:46.122124 23476 ts_tablet_manager.cc:1939] T ae057fd3d8224beb9b03292a9358a288 P 872da1bf65c242c6a9e85b515e34cb9f: Deleting consensus metadata
I20260810 00:57:46.122380 22779 catalog_manager.cc:4841] TS 492165fc756d4ec7a9b0d43f9624f6dc (127.20.81.129:39145): tablet ae057fd3d8224beb9b03292a9358a288 (table test-table [id=135d50101a8d46dbb730bdc9e3e886c8]) successfully deleted
I20260810 00:57:46.123303 22779 catalog_manager.cc:4841] TS 872da1bf65c242c6a9e85b515e34cb9f (127.20.81.130:41307): tablet ae057fd3d8224beb9b03292a9358a288 (table test-table [id=135d50101a8d46dbb730bdc9e3e886c8]) successfully deleted
Aug 10 00:57:46 dist-test-slave-gp6n krb5kdc[22743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323466, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 00:57:46 dist-test-slave-gp6n krb5kdc[22743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323466, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:46.167119 23484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:46.147690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41104 (local address 127.20.81.190:38411)
0810 00:57:46.148622 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:46.148627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:46.148648 (+ 21us) server_negotiation.cc:407] Connection header received
0810 00:57:46.148704 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:46.148708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:46.148764 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:46.148865 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:46.150383 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:46.151232 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:46.152484 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:46.152864 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:46.152977 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:46.161710 (+ 8733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:46.161716 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:46.161720 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:46.161760 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:46.164523 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:46.165285 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:46.165289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:46.165292 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:46.165411 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:46.165821 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:46.165824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:46.165826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:46.166118 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:46.166303 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:46.166660 (+ 357us) server_negotiation.cc:299] Negotiation successful
0810 00:57:46.166840 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":830,"thread_start_us":644,"threads_started":1}
I20260810 00:57:46.171476 22793 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41104:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:57:46.172096 22793 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:57:46.178930 22949 tablet_service.cc:1467] Processing CreateTablet for tablet fb318db4930447969523cbe8014c8ec1 (DEFAULT_TABLE table=test-table [id=012ac7f5c6bb4b77903784eb669e85e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:57:46.179090 23109 tablet_service.cc:1467] Processing CreateTablet for tablet fb318db4930447969523cbe8014c8ec1 (DEFAULT_TABLE table=test-table [id=012ac7f5c6bb4b77903784eb669e85e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:57:46.179267 23266 tablet_service.cc:1467] Processing CreateTablet for tablet fb318db4930447969523cbe8014c8ec1 (DEFAULT_TABLE table=test-table [id=012ac7f5c6bb4b77903784eb669e85e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:57:46.179486 22949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb318db4930447969523cbe8014c8ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:46.179553 23109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb318db4930447969523cbe8014c8ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:46.179641 23266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb318db4930447969523cbe8014c8ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:46.187232 23489 tablet_bootstrap.cc:492] T fb318db4930447969523cbe8014c8ec1 P 492165fc756d4ec7a9b0d43f9624f6dc: Bootstrap starting.
I20260810 00:57:46.189960 23489 tablet_bootstrap.cc:654] T fb318db4930447969523cbe8014c8ec1 P 492165fc756d4ec7a9b0d43f9624f6dc: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:46.191174 23490 tablet_bootstrap.cc:492] T fb318db4930447969523cbe8014c8ec1 P 872da1bf65c242c6a9e85b515e34cb9f: Bootstrap starting.
I20260810 00:57:46.192947 23489 tablet_bootstrap.cc:492] T fb318db4930447969523cbe8014c8ec1 P 492165fc756d4ec7a9b0d43f9624f6dc: No bootstrap required, opened a new log
I20260810 00:57:46.193073 23490 tablet_bootstrap.cc:654] T fb318db4930447969523cbe8014c8ec1 P 872da1bf65c242c6a9e85b515e34cb9f: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:46.193125 23489 ts_tablet_manager.cc:1397] T fb318db4930447969523cbe8014c8ec1 P 492165fc756d4ec7a9b0d43f9624f6dc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 00:57:46.194105 23489 raft_consensus.cc:348] T fb318db4930447969523cbe8014c8ec1 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:46.194443 23489 raft_consensus.cc:374] T fb318db4930447969523cbe8014c8ec1 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:46.194568 23489 raft_consensus.cc:729] T fb318db4930447969523cbe8014c8ec1 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 492165fc756d4ec7a9b0d43f9624f6dc, State: Initialized, Role: FOLLOWER
I20260810 00:57:46.194988 23489 consensus_queue.cc:260] T fb318db4930447969523cbe8014c8ec1 P 492165fc756d4ec7a9b0d43f9624f6dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:46.195557 23490 tablet_bootstrap.cc:492] T fb318db4930447969523cbe8014c8ec1 P 872da1bf65c242c6a9e85b515e34cb9f: No bootstrap required, opened a new log
I20260810 00:57:46.195873 23489 ts_tablet_manager.cc:1428] T fb318db4930447969523cbe8014c8ec1 P 492165fc756d4ec7a9b0d43f9624f6dc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 00:57:46.195905 23490 ts_tablet_manager.cc:1397] T fb318db4930447969523cbe8014c8ec1 P 872da1bf65c242c6a9e85b515e34cb9f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 00:57:46.196717 23488 tablet_bootstrap.cc:492] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c: Bootstrap starting.
I20260810 00:57:46.196841 23490 raft_consensus.cc:348] T fb318db4930447969523cbe8014c8ec1 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:46.197333 23490 raft_consensus.cc:374] T fb318db4930447969523cbe8014c8ec1 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:46.197839 23490 raft_consensus.cc:729] T fb318db4930447969523cbe8014c8ec1 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872da1bf65c242c6a9e85b515e34cb9f, State: Initialized, Role: FOLLOWER
I20260810 00:57:46.198177 23490 consensus_queue.cc:260] T fb318db4930447969523cbe8014c8ec1 P 872da1bf65c242c6a9e85b515e34cb9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:46.198698 23488 tablet_bootstrap.cc:654] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:46.200311 23490 ts_tablet_manager.cc:1428] T fb318db4930447969523cbe8014c8ec1 P 872da1bf65c242c6a9e85b515e34cb9f: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260810 00:57:46.201290 23488 tablet_bootstrap.cc:492] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c: No bootstrap required, opened a new log
I20260810 00:57:46.201434 23488 ts_tablet_manager.cc:1397] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 00:57:46.202260 23488 raft_consensus.cc:348] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:46.202531 23488 raft_consensus.cc:374] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:46.202639 23488 raft_consensus.cc:729] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 294011f87b014d1682732a8b7dc02f0c, State: Initialized, Role: FOLLOWER
I20260810 00:57:46.202876 23488 consensus_queue.cc:260] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:46.203785 23488 ts_tablet_manager.cc:1428] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 00:57:46.228114 23495 raft_consensus.cc:482] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:57:46.228358 23495 raft_consensus.cc:504] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:46.229975 23495 leader_election.cc:288] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 492165fc756d4ec7a9b0d43f9624f6dc (127.20.81.129:39145), 872da1bf65c242c6a9e85b515e34cb9f (127.20.81.130:41307)
I20260810 00:57:46.241250 23498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:46.231108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:41307 (local address 127.20.81.131:52317)
0810 00:57:46.231799 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:46.231824 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:46.231984 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:46.235389 (+ 3405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:46.235394 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:46.235421 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:57:46.235876 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:46.235891 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:46.237252 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:46.237257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:46.238908 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:46.238921 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:46.240391 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:46.240398 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:46.240621 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:46.240657 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 00:57:46.240756 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 00:57:46.240890 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":306,"threads_started":1}
I20260810 00:57:46.241328 23499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:46.231242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:52317 (local address 127.20.81.130:41307)
0810 00:57:46.235019 (+ 3777us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:46.235028 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:46.235062 (+ 34us) server_negotiation.cc:407] Connection header received
0810 00:57:46.235125 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:46.235129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:46.235189 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:46.236066 (+ 877us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:57:46.236285 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:46.237118 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:46.239076 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:46.240202 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:46.240362 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:46.240427 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:46.240787 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 00:57:46.240917 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3625,"thread_start_us":3412,"threads_started":1}
I20260810 00:57:46.241972 23496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:46.230218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:39145 (local address 127.20.81.131:41473)
0810 00:57:46.230839 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:46.230857 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:46.231038 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:46.236062 (+ 5024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:46.236066 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:46.236093 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:57:46.236395 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:46.236403 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:46.237813 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:46.237817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:46.239641 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:46.239656 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:46.241055 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:46.241063 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:46.241310 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:46.241356 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 00:57:46.241464 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 00:57:46.241607 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":348,"threads_started":1}
I20260810 00:57:46.241930 23139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb318db4930447969523cbe8014c8ec1" candidate_uuid: "294011f87b014d1682732a8b7dc02f0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872da1bf65c242c6a9e85b515e34cb9f" is_pre_election: true
I20260810 00:57:46.242177 23139 raft_consensus.cc:2454] T fb318db4930447969523cbe8014c8ec1 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 294011f87b014d1682732a8b7dc02f0c in term 0.
I20260810 00:57:46.242547 23497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:46.230290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:41473 (local address 127.20.81.129:39145)
0810 00:57:46.235729 (+ 5439us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:46.235734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:46.235757 (+ 23us) server_negotiation.cc:407] Connection header received
0810 00:57:46.235830 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:46.235836 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:46.235913 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:46.236048 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:57:46.236580 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:46.237673 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:46.239812 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:46.240828 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:46.240985 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:46.241059 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:46.241886 (+ 827us) server_negotiation.cc:299] Negotiation successful
0810 00:57:46.242060 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5293,"thread_start_us":5140,"threads_started":1}
I20260810 00:57:46.242733 23215 leader_election.cc:302] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 294011f87b014d1682732a8b7dc02f0c, 872da1bf65c242c6a9e85b515e34cb9f; no voters:
I20260810 00:57:46.243141 23495 raft_consensus.cc:2781] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:57:46.243278 23495 raft_consensus.cc:482] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:57:46.243186 22977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb318db4930447969523cbe8014c8ec1" candidate_uuid: "294011f87b014d1682732a8b7dc02f0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" is_pre_election: true
I20260810 00:57:46.243402 23495 raft_consensus.cc:3037] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:46.243503 22977 raft_consensus.cc:2454] T fb318db4930447969523cbe8014c8ec1 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 294011f87b014d1682732a8b7dc02f0c in term 0.
I20260810 00:57:46.246088 23495 raft_consensus.cc:504] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:46.246937 23495 leader_election.cc:288] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c [CANDIDATE]: Term 1 election: Requested vote from peers 492165fc756d4ec7a9b0d43f9624f6dc (127.20.81.129:39145), 872da1bf65c242c6a9e85b515e34cb9f (127.20.81.130:41307)
I20260810 00:57:46.247263 22977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb318db4930447969523cbe8014c8ec1" candidate_uuid: "294011f87b014d1682732a8b7dc02f0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "492165fc756d4ec7a9b0d43f9624f6dc"
I20260810 00:57:46.247345 23139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb318db4930447969523cbe8014c8ec1" candidate_uuid: "294011f87b014d1682732a8b7dc02f0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872da1bf65c242c6a9e85b515e34cb9f"
I20260810 00:57:46.247498 22977 raft_consensus.cc:3037] T fb318db4930447969523cbe8014c8ec1 P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:46.247540 23139 raft_consensus.cc:3037] T fb318db4930447969523cbe8014c8ec1 P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:46.250819 22977 raft_consensus.cc:2454] T fb318db4930447969523cbe8014c8ec1 P 492165fc756d4ec7a9b0d43f9624f6dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 294011f87b014d1682732a8b7dc02f0c in term 1.
I20260810 00:57:46.250820 23139 raft_consensus.cc:2454] T fb318db4930447969523cbe8014c8ec1 P 872da1bf65c242c6a9e85b515e34cb9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 294011f87b014d1682732a8b7dc02f0c in term 1.
I20260810 00:57:46.251248 23215 leader_election.cc:302] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 294011f87b014d1682732a8b7dc02f0c, 872da1bf65c242c6a9e85b515e34cb9f; no voters:
I20260810 00:57:46.251571 23495 raft_consensus.cc:2781] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:57:46.252168 23495 raft_consensus.cc:686] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c [term 1 LEADER]: Becoming Leader. State: Replica: 294011f87b014d1682732a8b7dc02f0c, State: Running, Role: LEADER
I20260810 00:57:46.252607 23495 consensus_queue.cc:237] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:46.255964 22792 catalog_manager.cc:5495] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c reported cstate change: term changed from 0 to 1, leader changed from <none> to 294011f87b014d1682732a8b7dc02f0c (127.20.81.131). New cstate: current_term: 1 leader_uuid: "294011f87b014d1682732a8b7dc02f0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } health_report { overall_health: HEALTHY } } }
I20260810 00:57:46.283151 23503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:46.277402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48366 (local address 127.20.81.131:34777)
0810 00:57:46.277982 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:46.277987 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:46.278006 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:57:46.278081 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:46.278084 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:46.278152 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:46.278286 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:46.278792 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:46.279615 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:46.280831 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:46.281237 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:46.281326 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:46.281795 (+ 469us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:46.282349 (+ 554us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:46.282525 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:46.282661 (+ 136us) server_negotiation.cc:299] Negotiation successful
0810 00:57:46.282744 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":324,"threads_started":1}
I20260810 00:57:46.300107 23497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:46.294068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:33283 (local address 127.20.81.129:39145)
0810 00:57:46.294462 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:46.294471 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:46.294866 (+ 395us) server_negotiation.cc:407] Connection header received
0810 00:57:46.295118 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:46.295122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:46.295192 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:46.295295 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:46.295853 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:46.296661 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:46.298125 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:46.298680 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:46.298818 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:46.299321 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:46.299452 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:46.299564 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:46.299743 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 00:57:46.299827 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260810 00:57:46.300138 23507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:46.293840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:39145 (local address 127.20.81.131:33283)
0810 00:57:46.294755 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:46.294772 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:46.294966 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:46.295310 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:46.295315 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:46.295328 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:57:46.295696 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:46.295706 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:46.296805 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:46.296811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:46.297977 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:46.297989 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:46.298889 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:46.298902 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:46.299054 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:46.299112 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:57:46.299601 (+ 489us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:57:46.299612 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 00:57:46.299687 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 00:57:46.299788 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"thread_start_us":488,"threads_started":1}
I20260810 00:57:46.310271 22977 raft_consensus.cc:1261] T fb318db4930447969523cbe8014c8ec1 P 492165fc756d4ec7a9b0d43f9624f6dc [term 1 FOLLOWER]: Refusing update from remote peer 294011f87b014d1682732a8b7dc02f0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:57:46.310281 23139 raft_consensus.cc:1261] T fb318db4930447969523cbe8014c8ec1 P 872da1bf65c242c6a9e85b515e34cb9f [term 1 FOLLOWER]: Refusing update from remote peer 294011f87b014d1682732a8b7dc02f0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:57:46.311196 23500 consensus_queue.cc:1035] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:57:46.311580 23495 consensus_queue.cc:1035] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:57:46.352067 23503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:46.345869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:46113 (local address 127.20.81.131:34777)
0810 00:57:46.346040 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:46.346047 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:46.346289 (+ 242us) server_negotiation.cc:407] Connection header received
0810 00:57:46.346495 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:46.346501 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:46.346586 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:46.346689 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:46.347488 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:46.348324 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:46.350385 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:46.350836 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:46.350977 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:46.351392 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:46.351495 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:46.351592 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:46.351706 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 00:57:46.351784 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 00:57:46.352211 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:46.345495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:34777 (local address 127.20.81.129:46113)
0810 00:57:46.346176 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:46.346198 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:46.346333 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:46.346869 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:46.346874 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:46.346887 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:57:46.347273 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:46.347285 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:46.348462 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:46.348469 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:46.350238 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:46.350254 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:46.351033 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:46.351041 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:46.351181 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:46.351219 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:57:46.351595 (+ 376us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:57:46.351609 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 00:57:46.351688 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 00:57:46.351822 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":575,"threads_started":1}
I20260810 00:57:46.384227 22792 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:57:46.384491 22792 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:57:46.387629 22792 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0: Sending DeleteTablet for 3 replicas of tablet fb318db4930447969523cbe8014c8ec1
I20260810 00:57:46.388550 23266 tablet_service.cc:1514] Processing DeleteTablet for tablet fb318db4930447969523cbe8014c8ec1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:57:46 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:48356
I20260810 00:57:46.388581 23109 tablet_service.cc:1514] Processing DeleteTablet for tablet fb318db4930447969523cbe8014c8ec1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:57:46 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:45280
I20260810 00:57:46.388857 23475 tablet_replica.cc:330] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c: stopping tablet replica
I20260810 00:57:46.388907 22949 tablet_service.cc:1514] Processing DeleteTablet for tablet fb318db4930447969523cbe8014c8ec1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:57:46 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:45968
I20260810 00:57:46.389098 23475 raft_consensus.cc:2229] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:57:46.389235 23474 tablet_replica.cc:330] T fb318db4930447969523cbe8014c8ec1 P 492165fc756d4ec7a9b0d43f9624f6dc: stopping tablet replica
I20260810 00:57:46.389470 23474 raft_consensus.cc:2229] T fb318db4930447969523cbe8014c8ec1 P 492165fc756d4ec7a9b0d43f9624f6dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:57:46.389671 23474 raft_consensus.cc:2258] T fb318db4930447969523cbe8014c8ec1 P 492165fc756d4ec7a9b0d43f9624f6dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:57:46.389659 23475 raft_consensus.cc:2258] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:57:46.390004 23476 tablet_replica.cc:330] T fb318db4930447969523cbe8014c8ec1 P 872da1bf65c242c6a9e85b515e34cb9f: stopping tablet replica
I20260810 00:57:46.389806 22792 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41104:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 00:57:46.390251 23476 raft_consensus.cc:2229] T fb318db4930447969523cbe8014c8ec1 P 872da1bf65c242c6a9e85b515e34cb9f [term 1 FOLLOWER]: Raft consensus shutting down.
W20260810 00:57:46.390499 22792 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:57:46.390513 23476 raft_consensus.cc:2258] T fb318db4930447969523cbe8014c8ec1 P 872da1bf65c242c6a9e85b515e34cb9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:57:46.391050 23474 ts_tablet_manager.cc:1905] T fb318db4930447969523cbe8014c8ec1 P 492165fc756d4ec7a9b0d43f9624f6dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:57:46.394300 23475 ts_tablet_manager.cc:1905] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:57:46.396018 23476 ts_tablet_manager.cc:1905] T fb318db4930447969523cbe8014c8ec1 P 872da1bf65c242c6a9e85b515e34cb9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:57:46.398748 23474 ts_tablet_manager.cc:1918] T fb318db4930447969523cbe8014c8ec1 P 492165fc756d4ec7a9b0d43f9624f6dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:57:46.399001 23474 log.cc:1196] T fb318db4930447969523cbe8014c8ec1 P 492165fc756d4ec7a9b0d43f9624f6dc: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-0/wal/wals/fb318db4930447969523cbe8014c8ec1
I20260810 00:57:46.399580 23474 ts_tablet_manager.cc:1939] T fb318db4930447969523cbe8014c8ec1 P 492165fc756d4ec7a9b0d43f9624f6dc: Deleting consensus metadata
I20260810 00:57:46.401021 22779 catalog_manager.cc:4841] TS 492165fc756d4ec7a9b0d43f9624f6dc (127.20.81.129:39145): tablet fb318db4930447969523cbe8014c8ec1 (table test-table [id=012ac7f5c6bb4b77903784eb669e85e8]) successfully deleted
I20260810 00:57:46.405341 23475 ts_tablet_manager.cc:1918] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:57:46.405604 23475 log.cc:1196] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/fb318db4930447969523cbe8014c8ec1
I20260810 00:57:46.405707 23109 tablet_service.cc:1467] Processing CreateTablet for tablet 90a63dabe104485c91a6394a0747132d (DEFAULT_TABLE table=test-table [id=2e3a496628fa4e738d09a505de97436e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:57:46.405920 23266 tablet_service.cc:1467] Processing CreateTablet for tablet 90a63dabe104485c91a6394a0747132d (DEFAULT_TABLE table=test-table [id=2e3a496628fa4e738d09a505de97436e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:57:46.406224 23109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90a63dabe104485c91a6394a0747132d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:46.406363 23266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90a63dabe104485c91a6394a0747132d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:46.407459 23475 ts_tablet_manager.cc:1939] T fb318db4930447969523cbe8014c8ec1 P 294011f87b014d1682732a8b7dc02f0c: Deleting consensus metadata
I20260810 00:57:46.409108 22778 catalog_manager.cc:4841] TS 294011f87b014d1682732a8b7dc02f0c (127.20.81.131:34777): tablet fb318db4930447969523cbe8014c8ec1 (table test-table [id=012ac7f5c6bb4b77903784eb669e85e8]) successfully deleted
I20260810 00:57:46.413488 23476 ts_tablet_manager.cc:1918] T fb318db4930447969523cbe8014c8ec1 P 872da1bf65c242c6a9e85b515e34cb9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:57:46.413697 23476 log.cc:1196] T fb318db4930447969523cbe8014c8ec1 P 872da1bf65c242c6a9e85b515e34cb9f: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/fb318db4930447969523cbe8014c8ec1
I20260810 00:57:46.413926 22949 tablet_service.cc:1467] Processing CreateTablet for tablet 90a63dabe104485c91a6394a0747132d (DEFAULT_TABLE table=test-table [id=2e3a496628fa4e738d09a505de97436e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:57:46.414309 23476 ts_tablet_manager.cc:1939] T fb318db4930447969523cbe8014c8ec1 P 872da1bf65c242c6a9e85b515e34cb9f: Deleting consensus metadata
I20260810 00:57:46.416563 22949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90a63dabe104485c91a6394a0747132d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:46.417310 23488 tablet_bootstrap.cc:492] T 90a63dabe104485c91a6394a0747132d P 294011f87b014d1682732a8b7dc02f0c: Bootstrap starting.
I20260810 00:57:46.419394 23488 tablet_bootstrap.cc:654] T 90a63dabe104485c91a6394a0747132d P 294011f87b014d1682732a8b7dc02f0c: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:46.423542 23489 tablet_bootstrap.cc:492] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc: Bootstrap starting.
I20260810 00:57:46.424010 22779 catalog_manager.cc:4841] TS 872da1bf65c242c6a9e85b515e34cb9f (127.20.81.130:41307): tablet fb318db4930447969523cbe8014c8ec1 (table test-table [id=012ac7f5c6bb4b77903784eb669e85e8]) successfully deleted
I20260810 00:57:46.425086 23490 tablet_bootstrap.cc:492] T 90a63dabe104485c91a6394a0747132d P 872da1bf65c242c6a9e85b515e34cb9f: Bootstrap starting.
I20260810 00:57:46.425271 23489 tablet_bootstrap.cc:654] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:46.427094 23490 tablet_bootstrap.cc:654] T 90a63dabe104485c91a6394a0747132d P 872da1bf65c242c6a9e85b515e34cb9f: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:46.430471 23488 tablet_bootstrap.cc:492] T 90a63dabe104485c91a6394a0747132d P 294011f87b014d1682732a8b7dc02f0c: No bootstrap required, opened a new log
I20260810 00:57:46.430651 23488 ts_tablet_manager.cc:1397] T 90a63dabe104485c91a6394a0747132d P 294011f87b014d1682732a8b7dc02f0c: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.003s
I20260810 00:57:46.431470 23488 raft_consensus.cc:348] T 90a63dabe104485c91a6394a0747132d P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:46.431739 23488 raft_consensus.cc:374] T 90a63dabe104485c91a6394a0747132d P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:46.431841 23488 raft_consensus.cc:729] T 90a63dabe104485c91a6394a0747132d P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 294011f87b014d1682732a8b7dc02f0c, State: Initialized, Role: FOLLOWER
I20260810 00:57:46.432090 23488 consensus_queue.cc:260] T 90a63dabe104485c91a6394a0747132d P 294011f87b014d1682732a8b7dc02f0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:46.432662 23488 ts_tablet_manager.cc:1428] T 90a63dabe104485c91a6394a0747132d P 294011f87b014d1682732a8b7dc02f0c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 00:57:46.439497 23490 tablet_bootstrap.cc:492] T 90a63dabe104485c91a6394a0747132d P 872da1bf65c242c6a9e85b515e34cb9f: No bootstrap required, opened a new log
I20260810 00:57:46.439670 23490 ts_tablet_manager.cc:1397] T 90a63dabe104485c91a6394a0747132d P 872da1bf65c242c6a9e85b515e34cb9f: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.009s
I20260810 00:57:46.440408 23490 raft_consensus.cc:348] T 90a63dabe104485c91a6394a0747132d P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:46.440685 23490 raft_consensus.cc:374] T 90a63dabe104485c91a6394a0747132d P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:46.440768 23490 raft_consensus.cc:729] T 90a63dabe104485c91a6394a0747132d P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872da1bf65c242c6a9e85b515e34cb9f, State: Initialized, Role: FOLLOWER
I20260810 00:57:46.441015 23490 consensus_queue.cc:260] T 90a63dabe104485c91a6394a0747132d P 872da1bf65c242c6a9e85b515e34cb9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:46.441587 23490 ts_tablet_manager.cc:1428] T 90a63dabe104485c91a6394a0747132d P 872da1bf65c242c6a9e85b515e34cb9f: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 00:57:46.441674 23489 tablet_bootstrap.cc:492] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc: No bootstrap required, opened a new log
I20260810 00:57:46.441859 23489 ts_tablet_manager.cc:1397] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.004s
I20260810 00:57:46.442929 23489 raft_consensus.cc:348] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:46.443397 23489 raft_consensus.cc:374] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:46.443632 23489 raft_consensus.cc:729] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 492165fc756d4ec7a9b0d43f9624f6dc, State: Initialized, Role: FOLLOWER
I20260810 00:57:46.444062 23489 consensus_queue.cc:260] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:46.446933 23489 ts_tablet_manager.cc:1428] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260810 00:57:46.451174 23454 raft_consensus.cc:482] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:57:46.451380 23454 raft_consensus.cc:504] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:46.452272 23454 leader_election.cc:288] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 872da1bf65c242c6a9e85b515e34cb9f (127.20.81.130:41307), 294011f87b014d1682732a8b7dc02f0c (127.20.81.131:34777)
I20260810 00:57:46.452908 23140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90a63dabe104485c91a6394a0747132d" candidate_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872da1bf65c242c6a9e85b515e34cb9f" is_pre_election: true
I20260810 00:57:46.453039 23292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90a63dabe104485c91a6394a0747132d" candidate_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "294011f87b014d1682732a8b7dc02f0c" is_pre_election: true
I20260810 00:57:46.453194 23140 raft_consensus.cc:2454] T 90a63dabe104485c91a6394a0747132d P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 492165fc756d4ec7a9b0d43f9624f6dc in term 0.
I20260810 00:57:46.453307 23292 raft_consensus.cc:2454] T 90a63dabe104485c91a6394a0747132d P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 492165fc756d4ec7a9b0d43f9624f6dc in term 0.
I20260810 00:57:46.453769 22895 leader_election.cc:302] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 492165fc756d4ec7a9b0d43f9624f6dc, 872da1bf65c242c6a9e85b515e34cb9f; no voters:
I20260810 00:57:46.454123 23454 raft_consensus.cc:2781] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:57:46.454247 23454 raft_consensus.cc:482] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:57:46.454331 23454 raft_consensus.cc:3037] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:46.457338 23454 raft_consensus.cc:504] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:46.458314 23140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90a63dabe104485c91a6394a0747132d" candidate_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872da1bf65c242c6a9e85b515e34cb9f"
I20260810 00:57:46.458546 23140 raft_consensus.cc:3037] T 90a63dabe104485c91a6394a0747132d P 872da1bf65c242c6a9e85b515e34cb9f [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:46.458618 23454 leader_election.cc:288] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc [CANDIDATE]: Term 1 election: Requested vote from peers 872da1bf65c242c6a9e85b515e34cb9f (127.20.81.130:41307), 294011f87b014d1682732a8b7dc02f0c (127.20.81.131:34777)
I20260810 00:57:46.459298 23292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90a63dabe104485c91a6394a0747132d" candidate_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "294011f87b014d1682732a8b7dc02f0c"
I20260810 00:57:46.459555 23292 raft_consensus.cc:3037] T 90a63dabe104485c91a6394a0747132d P 294011f87b014d1682732a8b7dc02f0c [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:46.461485 23140 raft_consensus.cc:2454] T 90a63dabe104485c91a6394a0747132d P 872da1bf65c242c6a9e85b515e34cb9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 492165fc756d4ec7a9b0d43f9624f6dc in term 1.
I20260810 00:57:46.461889 22895 leader_election.cc:302] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 492165fc756d4ec7a9b0d43f9624f6dc, 872da1bf65c242c6a9e85b515e34cb9f; no voters:
I20260810 00:57:46.462239 23454 raft_consensus.cc:2781] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:57:46.462735 23454 raft_consensus.cc:686] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc [term 1 LEADER]: Becoming Leader. State: Replica: 492165fc756d4ec7a9b0d43f9624f6dc, State: Running, Role: LEADER
I20260810 00:57:46.462970 23292 raft_consensus.cc:2454] T 90a63dabe104485c91a6394a0747132d P 294011f87b014d1682732a8b7dc02f0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 492165fc756d4ec7a9b0d43f9624f6dc in term 1.
I20260810 00:57:46.463020 23454 consensus_queue.cc:237] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } }
I20260810 00:57:46.466113 22791 catalog_manager.cc:5495] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 492165fc756d4ec7a9b0d43f9624f6dc (127.20.81.129). New cstate: current_term: 1 leader_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492165fc756d4ec7a9b0d43f9624f6dc" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 39145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 } health_report { overall_health: UNKNOWN } } }
I20260810 00:57:46.500891 23497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:46.494482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45992 (local address 127.20.81.129:39145)
0810 00:57:46.494665 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:46.494671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:46.494978 (+ 307us) server_negotiation.cc:407] Connection header received
0810 00:57:46.495151 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:46.495155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:46.495231 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:46.495333 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:46.495736 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:46.496590 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:46.498600 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:46.499169 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:46.499333 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:46.499906 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:46.500070 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:46.500161 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:46.500498 (+ 337us) server_negotiation.cc:299] Negotiation successful
0810 00:57:46.500594 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 00:57:46.511142 23292 raft_consensus.cc:1261] T 90a63dabe104485c91a6394a0747132d P 294011f87b014d1682732a8b7dc02f0c [term 1 FOLLOWER]: Refusing update from remote peer 492165fc756d4ec7a9b0d43f9624f6dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:57:46.511260 23140 raft_consensus.cc:1261] T 90a63dabe104485c91a6394a0747132d P 872da1bf65c242c6a9e85b515e34cb9f [term 1 FOLLOWER]: Refusing update from remote peer 492165fc756d4ec7a9b0d43f9624f6dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:57:46.511917 23454 consensus_queue.cc:1035] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "294011f87b014d1682732a8b7dc02f0c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 34777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:57:46.512233 23381 consensus_queue.cc:1035] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "872da1bf65c242c6a9e85b515e34cb9f" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 00:57:46.536530 23451 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 00:57:48.128368 22792 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:57:48.128633 22792 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:57:48.132691 22792 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c23651cd115445f8ac5de966480c07d0: Sending DeleteTablet for 3 replicas of tablet 90a63dabe104485c91a6394a0747132d
I20260810 00:57:48.133546 22949 tablet_service.cc:1514] Processing DeleteTablet for tablet 90a63dabe104485c91a6394a0747132d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:57:48 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:45968
I20260810 00:57:48.133694 23266 tablet_service.cc:1514] Processing DeleteTablet for tablet 90a63dabe104485c91a6394a0747132d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:57:48 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:48356
I20260810 00:57:48.133698 23109 tablet_service.cc:1514] Processing DeleteTablet for tablet 90a63dabe104485c91a6394a0747132d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:57:48 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:45280
I20260810 00:57:48.137428 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 22874
I20260810 00:57:48.144568 23550 tablet_replica.cc:330] T 90a63dabe104485c91a6394a0747132d P 294011f87b014d1682732a8b7dc02f0c: stopping tablet replica
I20260810 00:57:48.145006 23550 raft_consensus.cc:2229] T 90a63dabe104485c91a6394a0747132d P 294011f87b014d1682732a8b7dc02f0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:57:48.145414 23550 raft_consensus.cc:2258] T 90a63dabe104485c91a6394a0747132d P 294011f87b014d1682732a8b7dc02f0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:57:48.146168 23548 tablet_replica.cc:330] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc: stopping tablet replica
I20260810 00:57:48.146953 23550 ts_tablet_manager.cc:1905] T 90a63dabe104485c91a6394a0747132d P 294011f87b014d1682732a8b7dc02f0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:57:48.147280 23548 raft_consensus.cc:2229] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:57:48.147751 23548 raft_consensus.cc:2258] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:57:48.148661 23548 ts_tablet_manager.cc:1905] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:57:48.154021 23550 ts_tablet_manager.cc:1918] T 90a63dabe104485c91a6394a0747132d P 294011f87b014d1682732a8b7dc02f0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 00:57:48.154219 23550 log.cc:1196] T 90a63dabe104485c91a6394a0747132d P 294011f87b014d1682732a8b7dc02f0c: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/90a63dabe104485c91a6394a0747132d
I20260810 00:57:48.154780 23550 ts_tablet_manager.cc:1939] T 90a63dabe104485c91a6394a0747132d P 294011f87b014d1682732a8b7dc02f0c: Deleting consensus metadata
I20260810 00:57:48.155570 23548 ts_tablet_manager.cc:1918] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 00:57:48.155741 23548 log.cc:1196] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-0/wal/wals/90a63dabe104485c91a6394a0747132d
I20260810 00:57:48.155994 23549 tablet_replica.cc:330] T 90a63dabe104485c91a6394a0747132d P 872da1bf65c242c6a9e85b515e34cb9f: stopping tablet replica
I20260810 00:57:48.156227 23548 ts_tablet_manager.cc:1939] T 90a63dabe104485c91a6394a0747132d P 492165fc756d4ec7a9b0d43f9624f6dc: Deleting consensus metadata
I20260810 00:57:48.156169 22778 catalog_manager.cc:4841] TS 294011f87b014d1682732a8b7dc02f0c (127.20.81.131:34777): tablet 90a63dabe104485c91a6394a0747132d (table test-table [id=2e3a496628fa4e738d09a505de97436e]) successfully deleted
I20260810 00:57:48.156255 23549 raft_consensus.cc:2229] T 90a63dabe104485c91a6394a0747132d P 872da1bf65c242c6a9e85b515e34cb9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:57:48.156605 23549 raft_consensus.cc:2258] T 90a63dabe104485c91a6394a0747132d P 872da1bf65c242c6a9e85b515e34cb9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:57:48.157419 22779 catalog_manager.cc:4841] TS 492165fc756d4ec7a9b0d43f9624f6dc (127.20.81.129:39145): tablet 90a63dabe104485c91a6394a0747132d (table test-table [id=2e3a496628fa4e738d09a505de97436e]) successfully deleted
I20260810 00:57:48.157753 23549 ts_tablet_manager.cc:1905] T 90a63dabe104485c91a6394a0747132d P 872da1bf65c242c6a9e85b515e34cb9f: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 00:57:48 dist-test-slave-gp6n krb5kdc[22743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323466, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.81.129@KRBTEST.COM
I20260810 00:57:48.162513 23549 ts_tablet_manager.cc:1918] T 90a63dabe104485c91a6394a0747132d P 872da1bf65c242c6a9e85b515e34cb9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 00:57:48.162668 23549 log.cc:1196] T 90a63dabe104485c91a6394a0747132d P 872da1bf65c242c6a9e85b515e34cb9f: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/90a63dabe104485c91a6394a0747132d
I20260810 00:57:48.163084 23549 ts_tablet_manager.cc:1939] T 90a63dabe104485c91a6394a0747132d P 872da1bf65c242c6a9e85b515e34cb9f: Deleting consensus metadata
I20260810 00:57:48.163992 22779 catalog_manager.cc:4841] TS 872da1bf65c242c6a9e85b515e34cb9f (127.20.81.130:41307): tablet 90a63dabe104485c91a6394a0747132d (table test-table [id=2e3a496628fa4e738d09a505de97436e]) successfully deleted
I20260810 00:57:48.166611 23551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:48.138608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45994 (local address 127.20.81.129:39145)
0810 00:57:48.146885 (+ 8277us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:48.146894 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:48.146925 (+ 31us) server_negotiation.cc:407] Connection header received
0810 00:57:48.147002 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:48.147010 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:48.147117 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:48.147261 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:48.149633 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:48.151615 (+ 1982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:48.154189 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:48.155176 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:48.155351 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:48.160743 (+ 5392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:48.160757 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:48.160773 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:48.160804 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:48.163846 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:48.164603 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:48.164609 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:48.164612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:48.164737 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:48.165040 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:48.165042 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:48.165044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:48.165369 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:48.165687 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:48.166025 (+ 338us) server_negotiation.cc:299] Negotiation successful
0810 00:57:48.166236 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8103,"thread_start_us":7960,"threads_started":1}
W20260810 00:57:48.167243 23025 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45994
W20260810 00:57:48.167822 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 22874: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:57:48.168002 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 22874
I20260810 00:57:48.198381 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 23035
Aug 10 00:57:48 dist-test-slave-gp6n krb5kdc[22743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323466, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.81.130@KRBTEST.COM
I20260810 00:57:48.217584 23553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:48.199172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45292 (local address 127.20.81.130:41307)
0810 00:57:48.199803 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:48.199809 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:48.199829 (+ 20us) server_negotiation.cc:407] Connection header received
0810 00:57:48.199893 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:48.199898 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:48.199991 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:48.200102 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:48.202007 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:48.203184 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:48.204980 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:48.205605 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:48.205719 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:48.210869 (+ 5150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:48.210884 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:48.210900 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:48.210936 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:48.214349 (+ 3413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:48.215042 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:48.215050 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:48.215052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:48.215214 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:48.215592 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:48.215595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:48.215597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:48.216016 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:48.216481 (+ 465us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:48.216717 (+ 236us) server_negotiation.cc:299] Negotiation successful
0810 00:57:48.217079 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":347,"threads_started":1}
W20260810 00:57:48.218464 23182 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45292
W20260810 00:57:48.219223 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 23035: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:57:48.219404 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 23035
I20260810 00:57:48.248462 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 23194
Aug 10 00:57:48 dist-test-slave-gp6n krb5kdc[22743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323466, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.81.131@KRBTEST.COM
I20260810 00:57:48.263998 23554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:48.249240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48378 (local address 127.20.81.131:34777)
0810 00:57:48.249833 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:48.249838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:48.249854 (+ 16us) server_negotiation.cc:407] Connection header received
0810 00:57:48.249910 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:48.249913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:48.249973 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:48.250065 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:48.251432 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:48.252178 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:48.253432 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:48.253820 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:48.253934 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:48.258179 (+ 4245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:48.258192 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:48.258208 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:48.258244 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:48.261174 (+ 2930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:48.261860 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:48.261867 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:48.261870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:48.261996 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:48.262376 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:48.262383 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:48.262386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:48.262799 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:48.263119 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:48.263423 (+ 304us) server_negotiation.cc:299] Negotiation successful
0810 00:57:48.263647 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":310,"threads_started":1}
W20260810 00:57:48.264756 23347 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48378
W20260810 00:57:48.265399 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 23194: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:57:48.265591 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 23194
I20260810 00:57:48.290767 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 22758
I20260810 00:57:48.291997 22839 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:57:48.408927 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 22758
2026-08-10T00:57:48Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8666 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:57:48 dist-test-slave-gp6n krb5kdc[23558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:57:48 dist-test-slave-gp6n krb5kdc[23558](info): set up 2 sockets
Aug 10 00:57:48 dist-test-slave-gp6n krb5kdc[23558](info): commencing operation
krb5kdc: starting...
W20260810 00:57:49.591012 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 00:57:49 dist-test-slave-gp6n krb5kdc[23558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:57:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:57:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 00:57:49.814561 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:44579
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:40865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:44579
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:50.143837 23573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:50.144116 23573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:50.144284 23573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:50.151481 23573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:57:50.151593 23573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:50.151693 23573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:50.151788 23573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:57:50.151866 23573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:57:50.161682 23573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:40865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:44579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:44579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:50.163034 23573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:50.164455 23573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:50.176249 23582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:50.176306 23579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:50.176359 23578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:50.177426 23573 server_base.cc:1034] running on GCE node
I20260810 00:57:50.178194 23573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:50.179653 23573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:50.180891 23573 hybrid_clock.cc:648] HybridClock initialized: now 1786323470180830 us; error 53 us; skew 500 ppm
I20260810 00:57:50.181224 23573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:50 dist-test-slave-gp6n krb5kdc[23558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323470, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:50.184515 23573 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 00:57:50.185906 23573 webserver.cc:460] Webserver started at http://127.20.81.190:36499/ using document root <none> and password file <none>
I20260810 00:57:50.186664 23573 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:50.186811 23573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:50.187129 23573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:50.190037 23573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "f9bc9e21076a446e803d9ea2371d08f9"
format_stamp: "Formatted at 2026-08-10 00:57:50 on dist-test-slave-gp6n"
server_key: "948ba8106bc0441ea2d03bb58b7b8c2a"
server_key_iv: "57c1c468872154b8c1bd90c5b9134f4a"
server_key_version: "encryptionkey@0"
I20260810 00:57:50.190833 23573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "f9bc9e21076a446e803d9ea2371d08f9"
format_stamp: "Formatted at 2026-08-10 00:57:50 on dist-test-slave-gp6n"
server_key: "948ba8106bc0441ea2d03bb58b7b8c2a"
server_key_iv: "57c1c468872154b8c1bd90c5b9134f4a"
server_key_version: "encryptionkey@0"
I20260810 00:57:50.195856 23573 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 00:57:50.199242 23591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:50.200490 23573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 00:57:50.200677 23573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "f9bc9e21076a446e803d9ea2371d08f9"
format_stamp: "Formatted at 2026-08-10 00:57:50 on dist-test-slave-gp6n"
server_key: "948ba8106bc0441ea2d03bb58b7b8c2a"
server_key_iv: "57c1c468872154b8c1bd90c5b9134f4a"
server_key_version: "encryptionkey@0"
I20260810 00:57:50.200871 23573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:50.219951 23573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:50.225879 23573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:50.226203 23573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:50.251488 23573 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:44579
I20260810 00:57:50.251510 23651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:44579 every 8 connection(s)
I20260810 00:57:50.252921 23573 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:57:50.255757 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 23573
I20260810 00:57:50.256053 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 00:57:50.256969 20806 external_mini_cluster.cc:1428] Setting key bea1823a41ea6e3488fa119fa151a600
I20260810 00:57:50.257772 23652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 00:57:50 dist-test-slave-gp6n krb5kdc[23558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:50.273216 23652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9: Bootstrap starting.
I20260810 00:57:50.277868 23652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:50.279258 23652 log.cc:824] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:50.282955 23652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9: No bootstrap required, opened a new log
I20260810 00:57:50.283238 23655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:50.259205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49850 (local address 127.20.81.190:44579)
0810 00:57:50.260008 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:50.260025 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:50.260081 (+ 56us) server_negotiation.cc:407] Connection header received
0810 00:57:50.261958 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:50.261988 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:50.262583 (+ 595us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:50.263075 (+ 492us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:50.264397 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:50.266749 (+ 2352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:50.268056 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:50.269209 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:50.269442 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:50.273752 (+ 4310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:50.273763 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:50.273785 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:50.273830 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:50.278331 (+ 4501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:50.279144 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:50.279153 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:50.279162 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:50.279413 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:50.279875 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:50.279884 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:50.279890 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:50.280516 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:50.280813 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:50.281262 (+ 449us) server_negotiation.cc:299] Negotiation successful
0810 00:57:50.281726 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":340,"threads_started":1}
I20260810 00:57:50.291409 23652 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc9e21076a446e803d9ea2371d08f9" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44579 } }
I20260810 00:57:50.291761 23652 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:50.291850 23652 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9bc9e21076a446e803d9ea2371d08f9, State: Initialized, Role: FOLLOWER
I20260810 00:57:50.292533 23652 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc9e21076a446e803d9ea2371d08f9" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44579 } }
I20260810 00:57:50.292829 23652 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:57:50.293031 23652 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:57:50.293239 23652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:50.296067 23652 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc9e21076a446e803d9ea2371d08f9" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44579 } }
I20260810 00:57:50.296641 23652 leader_election.cc:302] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9bc9e21076a446e803d9ea2371d08f9; no voters:
I20260810 00:57:50.297256 23652 leader_election.cc:288] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:57:50.297431 23657 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:57:50.298157 23657 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9 [term 1 LEADER]: Becoming Leader. State: Replica: f9bc9e21076a446e803d9ea2371d08f9, State: Running, Role: LEADER
I20260810 00:57:50.298800 23657 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc9e21076a446e803d9ea2371d08f9" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44579 } }
I20260810 00:57:50.299865 23652 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:57:50.301898 23658 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9bc9e21076a446e803d9ea2371d08f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc9e21076a446e803d9ea2371d08f9" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44579 } } }
I20260810 00:57:50.301868 23659 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9bc9e21076a446e803d9ea2371d08f9. Latest consensus state: current_term: 1 leader_uuid: "f9bc9e21076a446e803d9ea2371d08f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc9e21076a446e803d9ea2371d08f9" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44579 } } }
I20260810 00:57:50.302367 23658 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9 [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:50.302500 23659 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9 [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:50.303208 23662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:57:50.310385 23662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:57:50.320075 23662 catalog_manager.cc:1347] Generated new cluster ID: 020515168f554273bc6f6c38747c255a
I20260810 00:57:50.320209 23662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:57:50.338315 23662 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:57:50.339973 23662 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:57:50.349501 23662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9bc9e21076a446e803d9ea2371d08f9: Generated new TSK 0
I20260810 00:57:50.350476 23662 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.81.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
I20260810 00:57:50.414716 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44579
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:40865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:50.762125 23679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:50.762490 23679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:50.762665 23679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:50.769953 23679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:50.770120 23679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:50.770296 23679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 00:57:50.780275 23679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:40865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:50.781646 23679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:50.783124 23679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:50.797118 23684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:50.797530 23687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:50.797583 23685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:50.798470 23679 server_base.cc:1034] running on GCE node
I20260810 00:57:50.799219 23679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:50.800418 23679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:50.801668 23679 hybrid_clock.cc:648] HybridClock initialized: now 1786323470801609 us; error 52 us; skew 500 ppm
I20260810 00:57:50.802047 23679 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:50 dist-test-slave-gp6n krb5kdc[23558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323470, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:50.806919 23679 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 00:57:50.808804 23679 webserver.cc:460] Webserver started at http://127.20.81.129:39009/ using document root <none> and password file <none>
I20260810 00:57:50.809755 23679 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:50.809929 23679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:50.810284 23679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:50.813522 23679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "a6c5d12fa905479a8d3eb9cde423c871"
format_stamp: "Formatted at 2026-08-10 00:57:50 on dist-test-slave-gp6n"
server_key: "5cbf12f7612752993f2e09583c9b286d"
server_key_iv: "e8e977347d0837d0f87c13831a0ef17d"
server_key_version: "encryptionkey@0"
I20260810 00:57:50.814558 23679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "a6c5d12fa905479a8d3eb9cde423c871"
format_stamp: "Formatted at 2026-08-10 00:57:50 on dist-test-slave-gp6n"
server_key: "5cbf12f7612752993f2e09583c9b286d"
server_key_iv: "e8e977347d0837d0f87c13831a0ef17d"
server_key_version: "encryptionkey@0"
I20260810 00:57:50.820549 23679 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260810 00:57:50.824558 23695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:50.826061 23679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 00:57:50.826278 23679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "a6c5d12fa905479a8d3eb9cde423c871"
format_stamp: "Formatted at 2026-08-10 00:57:50 on dist-test-slave-gp6n"
server_key: "5cbf12f7612752993f2e09583c9b286d"
server_key_iv: "e8e977347d0837d0f87c13831a0ef17d"
server_key_version: "encryptionkey@0"
I20260810 00:57:50.826680 23679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:50.853988 23679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:50.858752 23679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:50.859087 23679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:50.860272 23679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:57:50.861735 23679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:57:50.861848 23679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:50.862018 23679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:57:50.862099 23679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:50.900144 23679 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:43643
I20260810 00:57:50.900204 23823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:43643 every 8 connection(s)
I20260810 00:57:50.901831 23679 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 00:57:50.911512 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 23679
I20260810 00:57:50.911784 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 00:57:50.912462 20806 external_mini_cluster.cc:1428] Setting key 769538dd4b0d78b31504237216b10247
Aug 10 00:57:50 dist-test-slave-gp6n krb5kdc[23558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323470, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:50.924834 23826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:50.904451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:42259 (local address 127.20.81.190:44579)
0810 00:57:50.905002 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:50.905007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:50.906020 (+ 1013us) server_negotiation.cc:407] Connection header received
0810 00:57:50.908212 (+ 2192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:50.908219 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:50.908314 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:50.908461 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:50.910309 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:50.911364 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:50.913110 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:50.913695 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:50.913895 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:50.918292 (+ 4397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:50.918298 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:50.918301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:50.918347 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:50.921078 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:50.922020 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:50.922024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:50.922026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:50.922149 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:50.922680 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:50.922688 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:50.922694 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:50.923118 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:50.923311 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:50.923995 (+ 684us) server_negotiation.cc:299] Negotiation successful
0810 00:57:50.924267 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":418,"thread_start_us":280,"threads_started":1}
I20260810 00:57:50.925321 23827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:50.905089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44579 (local address 127.20.81.129:42259)
0810 00:57:50.905804 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:50.905856 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:50.907932 (+ 2076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:50.908741 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:50.908750 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:50.909470 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:50.910134 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:50.910153 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:50.911543 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:50.911547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:50.912963 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:50.912976 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:50.913958 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:50.913979 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:50.914323 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:50.915170 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:50.915195 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:50.918061 (+ 2866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:50.921227 (+ 3166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:50.921238 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:50.921257 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:50.921900 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:50.922302 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:50.922306 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:50.922308 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:50.922566 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:50.923259 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:50.923265 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:50.923594 (+ 329us) client_negotiation.cc:769] Sending connection context
0810 00:57:50.923811 (+ 217us) client_negotiation.cc:240] Negotiation successful
0810 00:57:50.924189 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":277,"threads_started":1}
I20260810 00:57:50.926894 23824 heartbeater.cc:347] Connected to a master server at 127.20.81.190:44579
I20260810 00:57:50.927204 23824 heartbeater.cc:464] Registering TS with master...
I20260810 00:57:50.927932 23824 heartbeater.cc:511] Master 127.20.81.190:44579 requested a full tablet report, sending...
I20260810 00:57:50.930058 23612 ts_manager.cc:194] Registered new tserver with Master: a6c5d12fa905479a8d3eb9cde423c871 (127.20.81.129:43643)
I20260810 00:57:50.931761 23612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:42259
WARNING: no policy specified for kudu/127.20.81.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.130@KRBTEST.COM" created.
I20260810 00:57:51.936653 23824 heartbeater.cc:503] Master 127.20.81.190:44579 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
W20260810 00:57:52.193342 20806 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.20.81.130: real 1.278s user 0.004s sys 0.011s
I20260810 00:57:52.194298 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44579
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:40865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:52.535740 23830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:52.535995 23830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:52.536087 23830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:52.542973 23830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:52.543080 23830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:52.543190 23830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 00:57:52.552495 23830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:40865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:52.553723 23830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:52.554998 23830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:52.568632 23836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:52.568608 23839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:52.568572 23837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:52.569341 23830 server_base.cc:1034] running on GCE node
I20260810 00:57:52.569921 23830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:52.570923 23830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:52.572165 23830 hybrid_clock.cc:648] HybridClock initialized: now 1786323472572104 us; error 60 us; skew 500 ppm
I20260810 00:57:52.572477 23830 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:52 dist-test-slave-gp6n krb5kdc[23558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323472, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:52.580811 23830 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 00:57:52.582195 23830 webserver.cc:460] Webserver started at http://127.20.81.130:36257/ using document root <none> and password file <none>
I20260810 00:57:52.582896 23830 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:52.583006 23830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:52.583269 23830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:52.586138 23830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "1a527c336b0d49138ac6c2cc51e23e08"
format_stamp: "Formatted at 2026-08-10 00:57:52 on dist-test-slave-gp6n"
server_key: "beea7c700f011dba90b4b48d6e2d029b"
server_key_iv: "9adb23fff312040cbcbd6ecca64ad86a"
server_key_version: "encryptionkey@0"
I20260810 00:57:52.587000 23830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "1a527c336b0d49138ac6c2cc51e23e08"
format_stamp: "Formatted at 2026-08-10 00:57:52 on dist-test-slave-gp6n"
server_key: "beea7c700f011dba90b4b48d6e2d029b"
server_key_iv: "9adb23fff312040cbcbd6ecca64ad86a"
server_key_version: "encryptionkey@0"
I20260810 00:57:52.592000 23830 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 00:57:52.595347 23847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:52.596611 23830 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 00:57:52.596755 23830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "1a527c336b0d49138ac6c2cc51e23e08"
format_stamp: "Formatted at 2026-08-10 00:57:52 on dist-test-slave-gp6n"
server_key: "beea7c700f011dba90b4b48d6e2d029b"
server_key_iv: "9adb23fff312040cbcbd6ecca64ad86a"
server_key_version: "encryptionkey@0"
I20260810 00:57:52.596926 23830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:52.607847 23830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:52.612080 23830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:52.612435 23830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:52.613437 23830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:57:52.614800 23830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:57:52.614877 23830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:52.615002 23830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:57:52.615052 23830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:52.646965 23830 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:33611
I20260810 00:57:52.647014 23977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:33611 every 8 connection(s)
I20260810 00:57:52.648265 23830 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 00:57:52.657734 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 23830
I20260810 00:57:52.657939 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 00:57:52.658551 20806 external_mini_cluster.cc:1428] Setting key 94c0565a252b3790ba9e9ea7440728b1
Aug 10 00:57:52 dist-test-slave-gp6n krb5kdc[23558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323472, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:52.668319 23981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:52.650973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:44237 (local address 127.20.81.190:44579)
0810 00:57:52.651463 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:52.651468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:52.652310 (+ 842us) server_negotiation.cc:407] Connection header received
0810 00:57:52.654151 (+ 1841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:52.654156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:52.654261 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:52.654382 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:52.655973 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:52.656728 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:52.658378 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:52.658901 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:52.659039 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:52.662660 (+ 3621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:52.662664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:52.662667 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:52.662698 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:52.665245 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:52.665973 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:52.665980 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:52.665983 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:52.666106 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:52.666580 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:52.666587 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:52.666592 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:52.666857 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:52.667019 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:52.667715 (+ 696us) server_negotiation.cc:299] Negotiation successful
0810 00:57:52.667954 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":272,"threads_started":1}
I20260810 00:57:52.669309 23983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:52.651407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44579 (local address 127.20.81.130:44237)
0810 00:57:52.652137 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:52.652175 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:52.653867 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:52.654634 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:52.654642 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:52.655249 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:52.655801 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:52.655817 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:52.656892 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:52.656896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:52.658236 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:52.658253 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:52.659131 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:52.659139 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:52.659363 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:52.660049 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:52.660070 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:52.662392 (+ 2322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:52.665393 (+ 3001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:52.665401 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:52.665425 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:52.665849 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:52.666220 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:52.666225 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:52.666229 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:52.666461 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:52.667002 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:52.667010 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:52.667348 (+ 338us) client_negotiation.cc:769] Sending connection context
0810 00:57:52.667613 (+ 265us) client_negotiation.cc:240] Negotiation successful
0810 00:57:52.668115 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":297,"threads_started":1}
I20260810 00:57:52.670806 23978 heartbeater.cc:347] Connected to a master server at 127.20.81.190:44579
I20260810 00:57:52.671139 23978 heartbeater.cc:464] Registering TS with master...
I20260810 00:57:52.672029 23978 heartbeater.cc:511] Master 127.20.81.190:44579 requested a full tablet report, sending...
I20260810 00:57:52.673682 23612 ts_manager.cc:194] Registered new tserver with Master: 1a527c336b0d49138ac6c2cc51e23e08 (127.20.81.130:33611)
I20260810 00:57:52.674567 23612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:44237
WARNING: no policy specified for kudu/127.20.81.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
I20260810 00:57:52.710920 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44579
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:40865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:53.041478 23986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:53.041922 23986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:53.042109 23986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:53.050377 23986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:53.050549 23986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:53.050757 23986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 00:57:53.060827 23986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:40865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:53.062266 23986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:53.063583 23986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:53.076534 23992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:53.076488 23993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:53.076571 23995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:53.077482 23986 server_base.cc:1034] running on GCE node
I20260810 00:57:53.078044 23986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:53.079046 23986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:53.080298 23986 hybrid_clock.cc:648] HybridClock initialized: now 1786323473080230 us; error 48 us; skew 500 ppm
I20260810 00:57:53.080644 23986 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:53 dist-test-slave-gp6n krb5kdc[23558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323473, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:53.083827 23986 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 00:57:53.085247 23986 webserver.cc:460] Webserver started at http://127.20.81.131:41501/ using document root <none> and password file <none>
I20260810 00:57:53.085883 23986 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:53.085983 23986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:53.086237 23986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:53.089138 23986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "6bb1b874b9904d89949ec2155b7077b5"
format_stamp: "Formatted at 2026-08-10 00:57:53 on dist-test-slave-gp6n"
server_key: "7dadac080c695fc75116135049674b97"
server_key_iv: "434a288929aabbc09514b7b8f0e9c9d3"
server_key_version: "encryptionkey@0"
I20260810 00:57:53.089898 23986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "6bb1b874b9904d89949ec2155b7077b5"
format_stamp: "Formatted at 2026-08-10 00:57:53 on dist-test-slave-gp6n"
server_key: "7dadac080c695fc75116135049674b97"
server_key_iv: "434a288929aabbc09514b7b8f0e9c9d3"
server_key_version: "encryptionkey@0"
I20260810 00:57:53.094941 23986 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260810 00:57:53.098237 24003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:53.099768 23986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:57:53.100004 23986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "6bb1b874b9904d89949ec2155b7077b5"
format_stamp: "Formatted at 2026-08-10 00:57:53 on dist-test-slave-gp6n"
server_key: "7dadac080c695fc75116135049674b97"
server_key_iv: "434a288929aabbc09514b7b8f0e9c9d3"
server_key_version: "encryptionkey@0"
I20260810 00:57:53.100207 23986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:53.112877 23986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:53.117126 23986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:53.117422 23986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:53.118378 23986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:57:53.119791 23986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:57:53.119903 23986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:53.120042 23986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:57:53.120117 23986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:53.152588 23986 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:33315
I20260810 00:57:53.152678 24133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:33315 every 8 connection(s)
I20260810 00:57:53.154134 23986 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 00:57:53.162812 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 23986
I20260810 00:57:53.163007 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 00:57:53.163491 20806 external_mini_cluster.cc:1428] Setting key 57878622264375ed7b3c397a634d61bd
Aug 10 00:57:53 dist-test-slave-gp6n krb5kdc[23558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323473, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:53.175551 23981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:53.157144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:33079 (local address 127.20.81.190:44579)
0810 00:57:53.157348 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:53.157353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:53.158581 (+ 1228us) server_negotiation.cc:407] Connection header received
0810 00:57:53.160665 (+ 2084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:53.160672 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:53.160769 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:53.160892 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:53.162472 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:53.163471 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:53.165676 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:53.166087 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:53.166238 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:53.169953 (+ 3715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:53.169962 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:53.169968 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:53.170017 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:53.172826 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:53.173525 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:53.173528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:53.173530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:53.173640 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:53.174060 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:53.174063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:53.174065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:53.174310 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:53.174490 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:53.175148 (+ 658us) server_negotiation.cc:299] Negotiation successful
0810 00:57:53.175306 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 00:57:53.176519 24136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:53.157564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44579 (local address 127.20.81.131:33079)
0810 00:57:53.158368 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:53.158463 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:53.160381 (+ 1918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:53.161141 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:53.161149 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:53.161685 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:53.162267 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:53.162285 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:53.163631 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:53.163639 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:53.165535 (+ 1896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:53.165552 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:53.166381 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:53.166392 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:53.166696 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:53.167355 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:53.167387 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:53.169720 (+ 2333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:53.172975 (+ 3255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:53.172982 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:53.172994 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:53.173421 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:53.173768 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:53.173771 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:53.173774 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:53.173964 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:53.174515 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:53.174522 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:53.174819 (+ 297us) client_negotiation.cc:769] Sending connection context
0810 00:57:53.175054 (+ 235us) client_negotiation.cc:240] Negotiation successful
0810 00:57:53.175436 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":355,"threads_started":1}
I20260810 00:57:53.178164 24134 heartbeater.cc:347] Connected to a master server at 127.20.81.190:44579
I20260810 00:57:53.178594 24134 heartbeater.cc:464] Registering TS with master...
I20260810 00:57:53.179510 24134 heartbeater.cc:511] Master 127.20.81.190:44579 requested a full tablet report, sending...
I20260810 00:57:53.181162 23610 ts_manager.cc:194] Registered new tserver with Master: 6bb1b874b9904d89949ec2155b7077b5 (127.20.81.131:33315)
I20260810 00:57:53.182122 23610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:33079
I20260810 00:57:53.191409 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 00:57:53.227936 23981 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:53.225310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45626 (local address 127.20.81.190:44579)
0810 00:57:53.225563 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:53.225571 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:53.225936 (+ 365us) server_negotiation.cc:407] Connection header received
0810 00:57:53.226073 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:53.226077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:53.226159 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:53.226249 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:53.227640 (+ 1391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45626: BlockingRecv error: recv got EOF from 127.0.0.1:45626 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 00:57:53.231439 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 23679
W20260810 00:57:53.234628 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 23679: Not authorized: Client connection negotiation failed: client connection to 127.20.81.129:43643: server requires authentication, but client does not have Kerberos credentials available
I20260810 00:57:53.234833 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 23679
I20260810 00:57:53.255371 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 23830
W20260810 00:57:53.258322 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 23830: Not authorized: Client connection negotiation failed: client connection to 127.20.81.130:33611: server requires authentication, but client does not have Kerberos credentials available
I20260810 00:57:53.258498 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 23830
I20260810 00:57:53.278594 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 23986
W20260810 00:57:53.281399 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 23986: Not authorized: Client connection negotiation failed: client connection to 127.20.81.131:33315: server requires authentication, but client does not have Kerberos credentials available
I20260810 00:57:53.281567 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 23986
I20260810 00:57:53.299899 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 23573
I20260810 00:57:53.301278 23647 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:57:53.404781 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 23573
2026-08-10T00:57:53Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4992 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:57:53 dist-test-slave-gp6n krb5kdc[24151](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:57:53 dist-test-slave-gp6n krb5kdc[24151](info): set up 2 sockets
Aug 10 00:57:53 dist-test-slave-gp6n krb5kdc[24151](info): commencing operation
krb5kdc: starting...
W20260810 00:57:54.587286 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.004s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 00:57:54 dist-test-slave-gp6n krb5kdc[24151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:57:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:57:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 00:57:54.813627 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:45873
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:45583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:45873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:55.149551 24165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:55.149928 24165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:55.150103 24165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:55.157373 24165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:57:55.157534 24165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:55.157650 24165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:55.157764 24165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:57:55.157859 24165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:57:55.167783 24165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:45583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:45873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:45873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:55.169180 24165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:55.170619 24165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:55.182986 24170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:55.183068 24173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:55.182976 24171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:55.184041 24165 server_base.cc:1034] running on GCE node
I20260810 00:57:55.184818 24165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:55.186180 24165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:55.187434 24165 hybrid_clock.cc:648] HybridClock initialized: now 1786323475187385 us; error 66 us; skew 500 ppm
I20260810 00:57:55.187781 24165 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:55 dist-test-slave-gp6n krb5kdc[24151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323475, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:55.191097 24165 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 00:57:55.192605 24165 webserver.cc:460] Webserver started at http://127.20.81.190:36203/ using document root <none> and password file <none>
I20260810 00:57:55.193295 24165 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:55.193415 24165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:55.193732 24165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:55.197121 24165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "5a6b3c828ec448a9b1e435f5216fa385"
format_stamp: "Formatted at 2026-08-10 00:57:55 on dist-test-slave-gp6n"
server_key: "b95ed31bdd5b196bc5a2849a4b71d28c"
server_key_iv: "8a1946282173821de32b100dd1e80cf6"
server_key_version: "encryptionkey@0"
I20260810 00:57:55.197995 24165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "5a6b3c828ec448a9b1e435f5216fa385"
format_stamp: "Formatted at 2026-08-10 00:57:55 on dist-test-slave-gp6n"
server_key: "b95ed31bdd5b196bc5a2849a4b71d28c"
server_key_iv: "8a1946282173821de32b100dd1e80cf6"
server_key_version: "encryptionkey@0"
I20260810 00:57:55.203579 24165 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 00:57:55.207274 24181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:55.208709 24165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 00:57:55.208891 24165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "5a6b3c828ec448a9b1e435f5216fa385"
format_stamp: "Formatted at 2026-08-10 00:57:55 on dist-test-slave-gp6n"
server_key: "b95ed31bdd5b196bc5a2849a4b71d28c"
server_key_iv: "8a1946282173821de32b100dd1e80cf6"
server_key_version: "encryptionkey@0"
I20260810 00:57:55.209110 24165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:55.221762 24165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:55.226102 24165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:55.226444 24165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:55.245124 24165 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:45873
I20260810 00:57:55.245148 24239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:45873 every 8 connection(s)
I20260810 00:57:55.246538 24165 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:57:55.250347 24240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:55.255353 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 24165
I20260810 00:57:55.255625 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 00:57:55.256269 20806 external_mini_cluster.cc:1428] Setting key 9374f931f7713341ef88aeb0615bf8a6
I20260810 00:57:55.262513 24240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385: Bootstrap starting.
I20260810 00:57:55.266227 24240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:55.267328 24240 log.cc:824] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:55.270390 24240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385: No bootstrap required, opened a new log
Aug 10 00:57:55 dist-test-slave-gp6n krb5kdc[24151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:55.280633 24240 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a6b3c828ec448a9b1e435f5216fa385" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 45873 } }
I20260810 00:57:55.281085 24240 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:55.281190 24240 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a6b3c828ec448a9b1e435f5216fa385, State: Initialized, Role: FOLLOWER
I20260810 00:57:55.281867 24240 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a6b3c828ec448a9b1e435f5216fa385" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 45873 } }
I20260810 00:57:55.282141 24240 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:57:55.282289 24240 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:57:55.282429 24243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:55.258370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43426 (local address 127.20.81.190:45873)
0810 00:57:55.259412 (+ 1042us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:55.259427 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:55.259477 (+ 50us) server_negotiation.cc:407] Connection header received
0810 00:57:55.261895 (+ 2418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:55.261925 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:55.262459 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:55.262911 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:55.264375 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:55.266396 (+ 2021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:55.267961 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:55.268665 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:55.268887 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:55.273249 (+ 4362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:55.273260 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:55.273282 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:55.273339 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:55.277571 (+ 4232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:55.278328 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:55.278337 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:55.278346 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:55.278586 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:55.278981 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:55.278986 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:55.278990 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:55.279863 (+ 873us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:55.280127 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:55.280546 (+ 419us) server_negotiation.cc:299] Negotiation successful
0810 00:57:55.281022 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":459,"threads_started":1}
I20260810 00:57:55.282588 24240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:55.285159 24240 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a6b3c828ec448a9b1e435f5216fa385" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 45873 } }
I20260810 00:57:55.285725 24240 leader_election.cc:302] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a6b3c828ec448a9b1e435f5216fa385; no voters:
I20260810 00:57:55.286489 24240 leader_election.cc:288] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:57:55.286720 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:57:55.286940 24245 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:57:55.288779 24240 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:57:55.295480 24245 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385 [term 1 LEADER]: Becoming Leader. State: Replica: 5a6b3c828ec448a9b1e435f5216fa385, State: Running, Role: LEADER
I20260810 00:57:55.296276 24245 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a6b3c828ec448a9b1e435f5216fa385" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 45873 } }
I20260810 00:57:55.299278 24246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a6b3c828ec448a9b1e435f5216fa385" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a6b3c828ec448a9b1e435f5216fa385" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 45873 } } }
I20260810 00:57:55.299672 24246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385 [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:55.300172 24247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a6b3c828ec448a9b1e435f5216fa385. Latest consensus state: current_term: 1 leader_uuid: "5a6b3c828ec448a9b1e435f5216fa385" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a6b3c828ec448a9b1e435f5216fa385" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 45873 } } }
I20260810 00:57:55.300443 24247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385 [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:55.302781 24260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:57:55.310015 24260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:57:55.321903 24269 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 00:57:55.329375 24260 catalog_manager.cc:1347] Generated new cluster ID: a6f6959ba60149418978a0a96ce72a76
I20260810 00:57:55.329521 24260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:57:55.340603 24260 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:57:55.342387 24260 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:57:55.344244 24243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:55.335912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43440 (local address 127.20.81.190:45873)
0810 00:57:55.336141 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:55.336151 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:55.336776 (+ 625us) server_negotiation.cc:407] Connection header received
0810 00:57:55.336996 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:55.337008 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:55.337078 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:55.337188 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:55.338215 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:55.339274 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:55.340827 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:55.341259 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:55.341337 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:55.342715 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:55.342720 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:55.342723 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:57:55.343366 (+ 643us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:55.343425 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:57:55.343455 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:55.343590 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:55.343706 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 00:57:55.343835 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":79}
I20260810 00:57:55.353438 24260 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a6b3c828ec448a9b1e435f5216fa385: Generated new TSK 0
I20260810 00:57:55.354456 24260 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:57:55.363088 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 24165
I20260810 00:57:55.364921 24234 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:57:55.454394 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 24165
2026-08-10T00:57:55Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2042 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:57:55 dist-test-slave-gp6n krb5kdc[24282](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:57:55 dist-test-slave-gp6n krb5kdc[24282](info): set up 2 sockets
Aug 10 00:57:55 dist-test-slave-gp6n krb5kdc[24282](info): commencing operation
krb5kdc: starting...
W20260810 00:57:56.721169 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.007s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 00:57:56 dist-test-slave-gp6n krb5kdc[24282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:57:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:57:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 00:57:56.947136 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:40541
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:34319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:40541
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:57.277025 24297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:57.277310 24297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:57.277405 24297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:57.284552 24297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:57:57.284680 24297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:57.284729 24297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:57.284780 24297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:57:57.284835 24297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:57:57.295024 24297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:34319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:40541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:40541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:57.296399 24297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:57.297734 24297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:57.309970 24303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:57.309993 24304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:57.310145 24306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:57.310735 24297 server_base.cc:1034] running on GCE node
I20260810 00:57:57.311642 24297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:57.313113 24297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:57.314462 24297 hybrid_clock.cc:648] HybridClock initialized: now 1786323477314366 us; error 59 us; skew 500 ppm
I20260810 00:57:57.314854 24297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:57 dist-test-slave-gp6n krb5kdc[24282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323477, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:57.318347 24297 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 00:57:57.319818 24297 webserver.cc:460] Webserver started at http://127.20.81.190:39409/ using document root <none> and password file <none>
I20260810 00:57:57.320501 24297 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:57.320652 24297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:57.320968 24297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:57.324196 24297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "fdc7264d34d84e8ea82da1ea3f45f12c"
format_stamp: "Formatted at 2026-08-10 00:57:57 on dist-test-slave-gp6n"
server_key: "ae386a94f118deeccb3289a66a83d9c9"
server_key_iv: "38844defe19d4eafc0693822b02997c6"
server_key_version: "encryptionkey@0"
I20260810 00:57:57.324990 24297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "fdc7264d34d84e8ea82da1ea3f45f12c"
format_stamp: "Formatted at 2026-08-10 00:57:57 on dist-test-slave-gp6n"
server_key: "ae386a94f118deeccb3289a66a83d9c9"
server_key_iv: "38844defe19d4eafc0693822b02997c6"
server_key_version: "encryptionkey@0"
I20260810 00:57:57.330272 24297 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 00:57:57.333652 24314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:57.334997 24297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:57:57.335179 24297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "fdc7264d34d84e8ea82da1ea3f45f12c"
format_stamp: "Formatted at 2026-08-10 00:57:57 on dist-test-slave-gp6n"
server_key: "ae386a94f118deeccb3289a66a83d9c9"
server_key_iv: "38844defe19d4eafc0693822b02997c6"
server_key_version: "encryptionkey@0"
I20260810 00:57:57.335429 24297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:57.359907 24297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:57.364403 24297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:57.364704 24297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:57.383970 24297 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:40541
I20260810 00:57:57.384003 24370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:40541 every 8 connection(s)
I20260810 00:57:57.385442 24297 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:57:57.388911 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 24297
I20260810 00:57:57.389163 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 00:57:57.389467 24371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:57.389906 20806 external_mini_cluster.cc:1428] Setting key 841240bedb32f4c6e118a38c40a9f3e3
I20260810 00:57:57.404275 24371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c: Bootstrap starting.
Aug 10 00:57:57 dist-test-slave-gp6n krb5kdc[24282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:57.407461 24371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:57.408530 24371 log.cc:824] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:57.411465 24371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c: No bootstrap required, opened a new log
I20260810 00:57:57.413252 24374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:57.392121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46638 (local address 127.20.81.190:40541)
0810 00:57:57.393029 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:57.393040 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:57.393081 (+ 41us) server_negotiation.cc:407] Connection header received
0810 00:57:57.394627 (+ 1546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:57.394648 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:57.395031 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:57.395392 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:57.397123 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:57.398490 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:57.400252 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:57.400995 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:57.401135 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:57.405919 (+ 4784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:57.405927 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:57.405942 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:57.405983 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:57.409222 (+ 3239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:57.409931 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:57.409944 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:57.409949 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:57.410112 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:57.410533 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:57.410539 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:57.410542 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:57.411058 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:57.411401 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:57.411827 (+ 426us) server_negotiation.cc:299] Negotiation successful
0810 00:57:57.412267 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":409,"threads_started":1}
I20260810 00:57:57.420018 24371 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc7264d34d84e8ea82da1ea3f45f12c" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40541 } }
I20260810 00:57:57.420393 24371 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:57.420500 24371 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdc7264d34d84e8ea82da1ea3f45f12c, State: Initialized, Role: FOLLOWER
I20260810 00:57:57.421025 24371 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc7264d34d84e8ea82da1ea3f45f12c" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40541 } }
I20260810 00:57:57.421253 24371 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:57:57.421399 24371 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:57:57.421569 24371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:57.424217 24371 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc7264d34d84e8ea82da1ea3f45f12c" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40541 } }
I20260810 00:57:57.424680 24371 leader_election.cc:302] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdc7264d34d84e8ea82da1ea3f45f12c; no voters:
I20260810 00:57:57.425295 24371 leader_election.cc:288] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:57:57.425448 24376 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:57:57.426167 24376 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c [term 1 LEADER]: Becoming Leader. State: Replica: fdc7264d34d84e8ea82da1ea3f45f12c, State: Running, Role: LEADER
I20260810 00:57:57.426726 24376 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc7264d34d84e8ea82da1ea3f45f12c" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40541 } }
I20260810 00:57:57.427417 24371 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:57:57.429663 24378 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdc7264d34d84e8ea82da1ea3f45f12c. Latest consensus state: current_term: 1 leader_uuid: "fdc7264d34d84e8ea82da1ea3f45f12c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc7264d34d84e8ea82da1ea3f45f12c" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40541 } } }
I20260810 00:57:57.429993 24378 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:57.430536 24377 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdc7264d34d84e8ea82da1ea3f45f12c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc7264d34d84e8ea82da1ea3f45f12c" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40541 } } }
I20260810 00:57:57.430881 24377 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:57.432273 24383 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:57:57.436583 24383 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:57:57.445895 24383 catalog_manager.cc:1347] Generated new cluster ID: eed80f0a16e84a7e9c5453343375ad1a
I20260810 00:57:57.446002 24383 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:57:57.455868 24383 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:57:57.457834 24383 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:57:57.471040 24383 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdc7264d34d84e8ea82da1ea3f45f12c: Generated new TSK 0
I20260810 00:57:57.471959 24383 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.81.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
I20260810 00:57:57.534196 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:40541
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:34319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:57.868081 24403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:57.868458 24403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:57.868649 24403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:57.875756 24403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:57.875854 24403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:57.876034 24403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 00:57:57.885524 24403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:34319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:40541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:57.886838 24403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:57.888118 24403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:57.901178 24411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:57.901217 24408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:57.901144 24409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:57.902158 24403 server_base.cc:1034] running on GCE node
I20260810 00:57:57.902872 24403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:57.904076 24403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:57.905331 24403 hybrid_clock.cc:648] HybridClock initialized: now 1786323477905236 us; error 78 us; skew 500 ppm
I20260810 00:57:57.905663 24403 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:57 dist-test-slave-gp6n krb5kdc[24282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323477, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:57.912493 24403 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 00:57:57.914124 24403 webserver.cc:460] Webserver started at http://127.20.81.129:42237/ using document root <none> and password file <none>
I20260810 00:57:57.915025 24403 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:57.915169 24403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:57.915505 24403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:57.918553 24403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "d32697182d984589a08729482d3a1e25"
format_stamp: "Formatted at 2026-08-10 00:57:57 on dist-test-slave-gp6n"
server_key: "77db9ba67a3143b9b38fa88b932f1b2c"
server_key_iv: "abb385c800e99332136f64b7ca3c9e1a"
server_key_version: "encryptionkey@0"
I20260810 00:57:57.919394 24403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "d32697182d984589a08729482d3a1e25"
format_stamp: "Formatted at 2026-08-10 00:57:57 on dist-test-slave-gp6n"
server_key: "77db9ba67a3143b9b38fa88b932f1b2c"
server_key_iv: "abb385c800e99332136f64b7ca3c9e1a"
server_key_version: "encryptionkey@0"
I20260810 00:57:57.924906 24403 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 00:57:57.928687 24420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:57.930073 24403 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 00:57:57.930279 24403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "d32697182d984589a08729482d3a1e25"
format_stamp: "Formatted at 2026-08-10 00:57:57 on dist-test-slave-gp6n"
server_key: "77db9ba67a3143b9b38fa88b932f1b2c"
server_key_iv: "abb385c800e99332136f64b7ca3c9e1a"
server_key_version: "encryptionkey@0"
I20260810 00:57:57.930548 24403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:57.963709 24403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:57.967967 24403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:57.968294 24403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:57.969338 24403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:57:57.971053 24403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:57:57.971154 24403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:57.971287 24403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:57:57.971338 24403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:58.005788 24403 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:36451
I20260810 00:57:58.005846 24546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:36451 every 8 connection(s)
I20260810 00:57:58.007208 24403 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 00:57:58.015332 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 24403
I20260810 00:57:58.015580 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 00:57:58.016403 20806 external_mini_cluster.cc:1428] Setting key 5df1b18c501b699399a582a1b9053106
Aug 10 00:57:58 dist-test-slave-gp6n krb5kdc[24282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323477, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:58.029104 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:58.009899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:59281 (local address 127.20.81.190:40541)
0810 00:57:58.010430 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:58.010439 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:58.011376 (+ 937us) server_negotiation.cc:407] Connection header received
0810 00:57:58.013159 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:58.013163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:58.013232 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:58.013332 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:58.014912 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:58.016145 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:58.018149 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:58.018926 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:58.019063 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:58.023035 (+ 3972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:58.023041 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:58.023045 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:58.023078 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:58.025832 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:58.026648 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:58.026659 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:58.026661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:58.026780 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:58.027242 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:58.027246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:58.027249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:58.027564 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:58.027725 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:58.028458 (+ 733us) server_negotiation.cc:299] Negotiation successful
0810 00:57:58.028684 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":232,"threads_started":1}
I20260810 00:57:58.030103 24550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:58.010505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:40541 (local address 127.20.81.129:59281)
0810 00:57:58.011194 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:58.011238 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:58.012905 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:58.013554 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:58.013561 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:58.014061 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:58.014704 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:58.014723 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:58.016360 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:58.016365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:58.017943 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:58.017963 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:58.019305 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:58.019311 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:58.019507 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:58.020200 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:58.020227 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:58.022807 (+ 2580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:58.025986 (+ 3179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:58.025995 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:58.026011 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:58.026519 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:58.026894 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:58.026902 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:58.026905 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:58.027118 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:58.027698 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:58.027705 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:58.028085 (+ 380us) client_negotiation.cc:769] Sending connection context
0810 00:57:58.028353 (+ 268us) client_negotiation.cc:240] Negotiation successful
0810 00:57:58.028935 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":283,"threads_started":1}
I20260810 00:57:58.031829 24547 heartbeater.cc:347] Connected to a master server at 127.20.81.190:40541
I20260810 00:57:58.032142 24547 heartbeater.cc:464] Registering TS with master...
I20260810 00:57:58.032882 24547 heartbeater.cc:511] Master 127.20.81.190:40541 requested a full tablet report, sending...
I20260810 00:57:58.035251 24332 ts_manager.cc:194] Registered new tserver with Master: d32697182d984589a08729482d3a1e25 (127.20.81.129:36451)
I20260810 00:57:58.036753 24332 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:59281
WARNING: no policy specified for kudu/127.20.81.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
I20260810 00:57:58.073199 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:40541
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:34319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:58.406672 24554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:58.406949 24554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:58.407111 24554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:58.414499 24554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:58.414603 24554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:58.414811 24554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 00:57:58.424417 24554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:34319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:40541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:58.425793 24554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:58.427191 24554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:58.441449 24560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:58.441480 24561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:58.441635 24563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:58.442479 24554 server_base.cc:1034] running on GCE node
I20260810 00:57:58.443150 24554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:58.444144 24554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:58.445405 24554 hybrid_clock.cc:648] HybridClock initialized: now 1786323478445361 us; error 52 us; skew 500 ppm
I20260810 00:57:58.445745 24554 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:58 dist-test-slave-gp6n krb5kdc[24282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323478, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:58.449105 24554 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 00:57:58.450714 24554 webserver.cc:460] Webserver started at http://127.20.81.130:45105/ using document root <none> and password file <none>
I20260810 00:57:58.451429 24554 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:58.451588 24554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:58.451925 24554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:58.455181 24554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "9683cb4be6e648d88e9b62aa421a5174"
format_stamp: "Formatted at 2026-08-10 00:57:58 on dist-test-slave-gp6n"
server_key: "0b79fb2253106d574449f37c5bd000fd"
server_key_iv: "6dbe8bdb5139f0c2b5cee4d6b1a229ad"
server_key_version: "encryptionkey@0"
I20260810 00:57:58.456017 24554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "9683cb4be6e648d88e9b62aa421a5174"
format_stamp: "Formatted at 2026-08-10 00:57:58 on dist-test-slave-gp6n"
server_key: "0b79fb2253106d574449f37c5bd000fd"
server_key_iv: "6dbe8bdb5139f0c2b5cee4d6b1a229ad"
server_key_version: "encryptionkey@0"
I20260810 00:57:58.463152 24554 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260810 00:57:58.467131 24571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:58.468595 24554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 00:57:58.468809 24554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "9683cb4be6e648d88e9b62aa421a5174"
format_stamp: "Formatted at 2026-08-10 00:57:58 on dist-test-slave-gp6n"
server_key: "0b79fb2253106d574449f37c5bd000fd"
server_key_iv: "6dbe8bdb5139f0c2b5cee4d6b1a229ad"
server_key_version: "encryptionkey@0"
I20260810 00:57:58.469036 24554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:58.481943 24554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:58.486332 24554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:58.486697 24554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:58.487704 24554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:57:58.489135 24554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:57:58.489250 24554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:58.489423 24554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:57:58.489504 24554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:58.525952 24554 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:44393
I20260810 00:57:58.526048 24698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:44393 every 8 connection(s)
I20260810 00:57:58.527452 24554 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 00:57:58.534258 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 24554
I20260810 00:57:58.534588 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 00:57:58.535108 20806 external_mini_cluster.cc:1428] Setting key 2153d108793a477d6e63d95671fa2ad7
Aug 10 00:57:58 dist-test-slave-gp6n krb5kdc[24282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323478, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:58.548888 24702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:58.530351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:46265 (local address 127.20.81.190:40541)
0810 00:57:58.530927 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:58.530932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:58.532013 (+ 1081us) server_negotiation.cc:407] Connection header received
0810 00:57:58.533908 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:58.533914 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:58.534006 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:58.534128 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:58.535913 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:58.536823 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:58.538544 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:58.539014 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:58.539157 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:58.542924 (+ 3767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:58.542930 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:58.542934 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:58.542968 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:58.545592 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:58.546367 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:58.546374 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:58.546382 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:58.546595 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:58.547059 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:58.547065 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:58.547073 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:58.547361 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:58.547534 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:58.548246 (+ 712us) server_negotiation.cc:299] Negotiation successful
0810 00:57:58.548491 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":433,"thread_start_us":285,"threads_started":1}
I20260810 00:57:58.550021 24703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:58.531079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:40541 (local address 127.20.81.130:46265)
0810 00:57:58.531821 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:58.531878 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:58.533590 (+ 1712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:58.534387 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:58.534435 (+ 48us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:58.535044 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:58.535740 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:58.535758 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:58.536976 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:58.536981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:58.538318 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:58.538330 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:58.539267 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:58.539276 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:58.539503 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:58.540279 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:58.540332 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:58.542673 (+ 2341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:58.545744 (+ 3071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:58.545753 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:58.545772 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:58.546228 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:58.546713 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:58.546719 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:58.546723 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:58.546932 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:58.547511 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:58.547519 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:58.547871 (+ 352us) client_negotiation.cc:769] Sending connection context
0810 00:57:58.548148 (+ 277us) client_negotiation.cc:240] Negotiation successful
0810 00:57:58.548709 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":296,"threads_started":1}
I20260810 00:57:58.551764 24700 heartbeater.cc:347] Connected to a master server at 127.20.81.190:40541
I20260810 00:57:58.552104 24700 heartbeater.cc:464] Registering TS with master...
I20260810 00:57:58.552848 24700 heartbeater.cc:511] Master 127.20.81.190:40541 requested a full tablet report, sending...
I20260810 00:57:58.554524 24332 ts_manager.cc:194] Registered new tserver with Master: 9683cb4be6e648d88e9b62aa421a5174 (127.20.81.130:44393)
I20260810 00:57:58.555447 24332 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:46265
WARNING: no policy specified for kudu/127.20.81.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
I20260810 00:57:58.589787 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:40541
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:34319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:58.927227 24706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:58.927534 24706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:58.927686 24706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:58.935164 24706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:58.935348 24706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:58.935581 24706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 00:57:58.946110 24706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:34319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:40541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:58.947539 24706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:58.948899 24706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:58.962774 24713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:58.962796 24714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:58.962860 24716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:58.963649 24706 server_base.cc:1034] running on GCE node
I20260810 00:57:58.964322 24706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:58.965304 24706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:58.966529 24706 hybrid_clock.cc:648] HybridClock initialized: now 1786323478966473 us; error 71 us; skew 500 ppm
I20260810 00:57:58.966861 24706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:58 dist-test-slave-gp6n krb5kdc[24282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323478, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:58.970213 24706 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 00:57:58.971982 24706 webserver.cc:460] Webserver started at http://127.20.81.131:36713/ using document root <none> and password file <none>
I20260810 00:57:58.972699 24706 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:58.972873 24706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:58.973187 24706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:58.976311 24706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "7de272c5c5154be5a9115a394a62c31c"
format_stamp: "Formatted at 2026-08-10 00:57:58 on dist-test-slave-gp6n"
server_key: "4a950e55b0ccfe6cc536e2147a088dbd"
server_key_iv: "dc34d9e3dff0766b0d8b4304b433d6d9"
server_key_version: "encryptionkey@0"
I20260810 00:57:58.977120 24706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "7de272c5c5154be5a9115a394a62c31c"
format_stamp: "Formatted at 2026-08-10 00:57:58 on dist-test-slave-gp6n"
server_key: "4a950e55b0ccfe6cc536e2147a088dbd"
server_key_iv: "dc34d9e3dff0766b0d8b4304b433d6d9"
server_key_version: "encryptionkey@0"
I20260810 00:57:58.982538 24706 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 00:57:58.986083 24723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:58.987445 24706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:57:58.987622 24706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "7de272c5c5154be5a9115a394a62c31c"
format_stamp: "Formatted at 2026-08-10 00:57:58 on dist-test-slave-gp6n"
server_key: "4a950e55b0ccfe6cc536e2147a088dbd"
server_key_iv: "dc34d9e3dff0766b0d8b4304b433d6d9"
server_key_version: "encryptionkey@0"
I20260810 00:57:58.987816 24706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:59.010183 24706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:59.014941 24706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:59.015280 24706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:59.016285 24706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:57:59.017701 24706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:57:59.017810 24706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:59.017962 24706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:57:59.018038 24706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:59.039983 24547 heartbeater.cc:503] Master 127.20.81.190:40541 was elected leader, sending a full tablet report...
I20260810 00:57:59.053766 24706 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:46443
I20260810 00:57:59.053869 24855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:46443 every 8 connection(s)
I20260810 00:57:59.055265 24706 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 00:57:59.063535 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 24706
I20260810 00:57:59.063768 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 00:57:59.064241 20806 external_mini_cluster.cc:1428] Setting key 60bf247f9ae6d446ef1cc83e5022a797
Aug 10 00:57:59 dist-test-slave-gp6n krb5kdc[24282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323478, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:59.076225 24858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:59.057887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:35953 (local address 127.20.81.190:40541)
0810 00:57:59.058340 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:59.058348 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:59.059416 (+ 1068us) server_negotiation.cc:407] Connection header received
0810 00:57:59.061174 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:59.061178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:59.061248 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:59.061347 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:59.062914 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:59.063811 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:59.065592 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:59.065985 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:59.066118 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:59.070051 (+ 3933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:59.070059 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:59.070064 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:59.070138 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:59.073370 (+ 3232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:59.074074 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:59.074078 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:59.074081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:59.074231 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:59.074709 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:59.074713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:59.074714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:59.074979 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:59.075116 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:59.075763 (+ 647us) server_negotiation.cc:299] Negotiation successful
0810 00:57:59.075935 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":211,"threads_started":1}
I20260810 00:57:59.077211 24859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:59.058506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:40541 (local address 127.20.81.131:35953)
0810 00:57:59.059224 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:59.059263 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:59.060910 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:59.061571 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:59.061578 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:59.062082 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:59.062753 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:59.062771 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:59.064004 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:59.064008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:59.065431 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:59.065443 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:59.066218 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:59.066226 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:59.066609 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:59.067413 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:59.067435 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:59.069755 (+ 2320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:59.073495 (+ 3740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:59.073501 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:59.073516 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:59.073962 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:59.074370 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:59.074373 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:59.074375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:59.074603 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:59.075102 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:59.075108 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:59.075436 (+ 328us) client_negotiation.cc:769] Sending connection context
0810 00:57:59.075661 (+ 225us) client_negotiation.cc:240] Negotiation successful
0810 00:57:59.076058 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":288,"threads_started":1}
I20260810 00:57:59.078810 24856 heartbeater.cc:347] Connected to a master server at 127.20.81.190:40541
I20260810 00:57:59.079133 24856 heartbeater.cc:464] Registering TS with master...
I20260810 00:57:59.080093 24856 heartbeater.cc:511] Master 127.20.81.190:40541 requested a full tablet report, sending...
I20260810 00:57:59.081519 24332 ts_manager.cc:194] Registered new tserver with Master: 7de272c5c5154be5a9115a394a62c31c (127.20.81.131:46443)
I20260810 00:57:59.082314 24332 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:35953
I20260810 00:57:59.092540 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 00:57:59 dist-test-slave-gp6n krb5kdc[24282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323479, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 00:57:59 dist-test-slave-gp6n krb5kdc[24282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323479, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:57:59.146306 24858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:59.131421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46654 (local address 127.20.81.190:40541)
0810 00:57:59.131671 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:59.131680 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:59.132043 (+ 363us) server_negotiation.cc:407] Connection header received
0810 00:57:59.132267 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:59.132271 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:59.132340 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:59.132489 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:59.133906 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:59.134761 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:59.136087 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:59.136524 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:59.136663 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:59.140793 (+ 4130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:59.140799 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:59.140803 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:59.140836 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:59.143661 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:59.144312 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:59.144319 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:59.144322 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:59.144489 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:59.144870 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:59.144877 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:59.144882 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:59.145200 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:59.145450 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:59.145746 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 00:57:59.145980 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260810 00:57:59.146924 24332 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46654
I20260810 00:57:59.148977 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 24403
Aug 10 00:57:59 dist-test-slave-gp6n krb5kdc[24282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323479, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.81.129@KRBTEST.COM
I20260810 00:57:59.165112 24869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:59.150122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36386 (local address 127.20.81.129:36451)
0810 00:57:59.150824 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:59.150832 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:59.150858 (+ 26us) server_negotiation.cc:407] Connection header received
0810 00:57:59.150911 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:59.150918 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:59.151126 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:59.151248 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:59.152498 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:59.153495 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:59.154913 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:59.155468 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:59.155572 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:59.159768 (+ 4196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:59.159779 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:59.159791 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:59.159830 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:59.162619 (+ 2789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:59.163243 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:59.163250 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:59.163253 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:59.163402 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:59.163724 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:59.163728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:59.163730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:59.164058 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:59.164280 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:59.164531 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 00:57:59.164771 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":393,"threads_started":1}
W20260810 00:57:59.166545 24540 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36386
W20260810 00:57:59.167701 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 24403: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:57:59.167860 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 24403
I20260810 00:57:59.186904 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 24554
Aug 10 00:57:59 dist-test-slave-gp6n krb5kdc[24282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323479, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.81.130@KRBTEST.COM
I20260810 00:57:59.202821 24872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:59.187838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34624 (local address 127.20.81.130:44393)
0810 00:57:59.188434 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:59.188442 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:59.188467 (+ 25us) server_negotiation.cc:407] Connection header received
0810 00:57:59.188543 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:59.188549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:59.188766 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:59.188894 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:59.190180 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:59.191334 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:59.192703 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:59.193303 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:59.193408 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:59.197528 (+ 4120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:59.197539 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:59.197553 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:59.197586 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:59.200347 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:59.200932 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:59.200940 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:59.200943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:59.201073 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:59.201393 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:59.201396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:59.201398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:59.201737 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:59.201966 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:59.202238 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 00:57:59.202536 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":300,"threads_started":1}
W20260810 00:57:59.204149 24694 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34624
W20260810 00:57:59.205178 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 24554: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:57:59.205335 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 24554
I20260810 00:57:59.223555 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 24706
Aug 10 00:57:59 dist-test-slave-gp6n krb5kdc[24282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323479, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.81.131@KRBTEST.COM
I20260810 00:57:59.241003 24873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:59.224510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59074 (local address 127.20.81.131:46443)
0810 00:57:59.225143 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:59.225151 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:59.225178 (+ 27us) server_negotiation.cc:407] Connection header received
0810 00:57:59.225236 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:59.225241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:59.225460 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:59.225629 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:59.227113 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:59.228771 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:59.230053 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:59.230784 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:59.230946 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:59.235308 (+ 4362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:59.235319 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:59.235332 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:59.235370 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:59.238031 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:59.238870 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:59.238876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:59.238879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:59.239020 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:59.239399 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:59.239410 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:59.239413 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:59.239752 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:59.240077 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:59.240366 (+ 289us) server_negotiation.cc:299] Negotiation successful
0810 00:57:59.240650 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":330,"threads_started":1}
W20260810 00:57:59.242265 24848 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59074
W20260810 00:57:59.243467 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 24706: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:57:59.243637 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 24706
I20260810 00:57:59.263345 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 24297
I20260810 00:57:59.264653 24365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:57:59.368557 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 24297
2026-08-10T00:57:59Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3921 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:57:59 dist-test-slave-gp6n krb5kdc[24877](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:57:59 dist-test-slave-gp6n krb5kdc[24877](info): set up 2 sockets
Aug 10 00:57:59 dist-test-slave-gp6n krb5kdc[24877](info): commencing operation
krb5kdc: starting...
W20260810 00:58:00.583565 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.001s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 00:58:00 dist-test-slave-gp6n krb5kdc[24877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:58:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:58:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 00:58:00.816278 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:45973
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:39491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:45973
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:01.149641 24893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:01.149995 24893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:01.150170 24893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:01.157872 24893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:58:01.158067 24893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:01.158161 24893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:01.158277 24893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:58:01.158375 24893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:58:01.168655 24893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:39491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:45973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:45973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:01.170106 24893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:01.171875 24893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:01.183830 24898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:01.183827 24899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:01.184597 24901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:01.184911 24893 server_base.cc:1034] running on GCE node
I20260810 00:58:01.185719 24893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:01.187255 24893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:01.188529 24893 hybrid_clock.cc:648] HybridClock initialized: now 1786323481188497 us; error 62 us; skew 500 ppm
I20260810 00:58:01.188877 24893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:01 dist-test-slave-gp6n krb5kdc[24877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323481, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:01.192098 24893 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 00:58:01.193725 24893 webserver.cc:460] Webserver started at http://127.20.81.190:33643/ using document root <none> and password file <none>
I20260810 00:58:01.194474 24893 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:01.194620 24893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:01.194934 24893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:01.197885 24893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "997c55d0fab749e59027be9a108f0500"
format_stamp: "Formatted at 2026-08-10 00:58:01 on dist-test-slave-gp6n"
server_key: "f44f657346653768c5e6c8b006f1a9bb"
server_key_iv: "3cd6c60544e4ffd022faa9b4142b789b"
server_key_version: "encryptionkey@0"
I20260810 00:58:01.198727 24893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "997c55d0fab749e59027be9a108f0500"
format_stamp: "Formatted at 2026-08-10 00:58:01 on dist-test-slave-gp6n"
server_key: "f44f657346653768c5e6c8b006f1a9bb"
server_key_iv: "3cd6c60544e4ffd022faa9b4142b789b"
server_key_version: "encryptionkey@0"
I20260810 00:58:01.203600 24893 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 00:58:01.206967 24908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:01.208326 24893 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 00:58:01.208534 24893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "997c55d0fab749e59027be9a108f0500"
format_stamp: "Formatted at 2026-08-10 00:58:01 on dist-test-slave-gp6n"
server_key: "f44f657346653768c5e6c8b006f1a9bb"
server_key_iv: "3cd6c60544e4ffd022faa9b4142b789b"
server_key_version: "encryptionkey@0"
I20260810 00:58:01.208725 24893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:01.253084 24893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:01.257490 24893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:01.257800 24893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:01.277073 24893 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:45973
I20260810 00:58:01.277109 24966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:45973 every 8 connection(s)
I20260810 00:58:01.278473 24893 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:58:01.279485 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 24893
I20260810 00:58:01.279711 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 00:58:01.280396 20806 external_mini_cluster.cc:1428] Setting key de654f596c4f1d42efcce29a2cdb8391
I20260810 00:58:01.283067 24967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 00:58:01 dist-test-slave-gp6n krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:58:01.296205 24967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500: Bootstrap starting.
I20260810 00:58:01.300321 24967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:01.301585 24967 log.cc:824] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:01.305074 24967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500: No bootstrap required, opened a new log
I20260810 00:58:01.305613 24970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:01.282522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43104 (local address 127.20.81.190:45973)
0810 00:58:01.283334 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:01.283355 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:01.283411 (+ 56us) server_negotiation.cc:407] Connection header received
0810 00:58:01.285872 (+ 2461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:01.285908 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:01.286448 (+ 540us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:01.286913 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:01.288400 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:01.290532 (+ 2132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:01.291809 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:01.292828 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:01.293033 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:01.296996 (+ 3963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:01.297006 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:01.297022 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:01.297069 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:01.301107 (+ 4038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:01.301855 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:01.301862 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:01.301867 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:01.302077 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:01.302561 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:01.302569 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:01.302575 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:01.303253 (+ 678us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:01.303472 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:01.304049 (+ 577us) server_negotiation.cc:299] Negotiation successful
0810 00:58:01.304413 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":321,"threads_started":1}
I20260810 00:58:01.315275 24967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "997c55d0fab749e59027be9a108f0500" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 45973 } }
I20260810 00:58:01.315698 24967 raft_consensus.cc:374] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:01.315819 24967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997c55d0fab749e59027be9a108f0500, State: Initialized, Role: FOLLOWER
I20260810 00:58:01.316426 24967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "997c55d0fab749e59027be9a108f0500" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 45973 } }
I20260810 00:58:01.316686 24967 raft_consensus.cc:388] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:58:01.316833 24967 raft_consensus.cc:482] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:58:01.317040 24967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:01.319540 24967 raft_consensus.cc:504] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "997c55d0fab749e59027be9a108f0500" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 45973 } }
I20260810 00:58:01.320111 24967 leader_election.cc:302] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 997c55d0fab749e59027be9a108f0500; no voters:
I20260810 00:58:01.320776 24967 leader_election.cc:288] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:58:01.320869 24973 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:58:01.321731 24973 raft_consensus.cc:686] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500 [term 1 LEADER]: Becoming Leader. State: Replica: 997c55d0fab749e59027be9a108f0500, State: Running, Role: LEADER
I20260810 00:58:01.322440 24973 consensus_queue.cc:237] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "997c55d0fab749e59027be9a108f0500" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 45973 } }
I20260810 00:58:01.323372 24967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:58:01.325466 24975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 997c55d0fab749e59027be9a108f0500. Latest consensus state: current_term: 1 leader_uuid: "997c55d0fab749e59027be9a108f0500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "997c55d0fab749e59027be9a108f0500" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 45973 } } }
I20260810 00:58:01.325999 24975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:01.325824 24974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "997c55d0fab749e59027be9a108f0500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "997c55d0fab749e59027be9a108f0500" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 45973 } } }
I20260810 00:58:01.326149 24974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:01.328112 24982 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:58:01.333513 24982 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:58:01.343096 24982 catalog_manager.cc:1347] Generated new cluster ID: cf1261da432943c6839f681be2ae0988
I20260810 00:58:01.343237 24982 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:58:01.351143 24982 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:58:01.352468 24982 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:58:01.360337 24982 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 997c55d0fab749e59027be9a108f0500: Generated new TSK 0
I20260810 00:58:01.361193 24982 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.81.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
I20260810 00:58:01.422585 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:45973
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:39491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:01.757768 24996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:01.758064 24996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:01.758230 24996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:01.765463 24996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:01.765571 24996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:01.765756 24996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 00:58:01.775460 24996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:39491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:45973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:01.776803 24996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:01.778162 24996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:01.791443 25001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:01.791466 25002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:01.791390 25004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:01.792435 24996 server_base.cc:1034] running on GCE node
I20260810 00:58:01.792991 24996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:01.793864 24996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:01.795101 24996 hybrid_clock.cc:648] HybridClock initialized: now 1786323481795037 us; error 53 us; skew 500 ppm
I20260810 00:58:01.795488 24996 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:01 dist-test-slave-gp6n krb5kdc[24877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323481, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:01.799129 24996 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 00:58:01.800575 24996 webserver.cc:460] Webserver started at http://127.20.81.129:37311/ using document root <none> and password file <none>
I20260810 00:58:01.801292 24996 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:01.801427 24996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:01.801746 24996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:01.804761 24996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "afd6e90ae2b8440db791e15f8b7af3c4"
format_stamp: "Formatted at 2026-08-10 00:58:01 on dist-test-slave-gp6n"
server_key: "d9853f4fa3cf34a41c0afba78dfeb20c"
server_key_iv: "4644694f2d9cd7e6f89f78ba07a4961a"
server_key_version: "encryptionkey@0"
I20260810 00:58:01.805562 24996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "afd6e90ae2b8440db791e15f8b7af3c4"
format_stamp: "Formatted at 2026-08-10 00:58:01 on dist-test-slave-gp6n"
server_key: "d9853f4fa3cf34a41c0afba78dfeb20c"
server_key_iv: "4644694f2d9cd7e6f89f78ba07a4961a"
server_key_version: "encryptionkey@0"
I20260810 00:58:01.810708 24996 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 00:58:01.814185 25013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:01.815644 24996 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:58:01.815852 24996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "afd6e90ae2b8440db791e15f8b7af3c4"
format_stamp: "Formatted at 2026-08-10 00:58:01 on dist-test-slave-gp6n"
server_key: "d9853f4fa3cf34a41c0afba78dfeb20c"
server_key_iv: "4644694f2d9cd7e6f89f78ba07a4961a"
server_key_version: "encryptionkey@0"
I20260810 00:58:01.816079 24996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:01.838024 24996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:01.842363 24996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:01.842763 24996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:01.843850 24996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:01.845289 24996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:01.845367 24996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:01.845520 24996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:01.845575 24996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:01.879904 24996 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:33987
I20260810 00:58:01.879958 25143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:33987 every 8 connection(s)
I20260810 00:58:01.881232 24996 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 00:58:01.886018 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 24996
I20260810 00:58:01.886229 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 00:58:01.886950 20806 external_mini_cluster.cc:1428] Setting key f3af156589e51e8e3620d18da7d49826
Aug 10 00:58:01 dist-test-slave-gp6n krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323481, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:58:01.902932 25146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:01.883810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:34219 (local address 127.20.81.190:45973)
0810 00:58:01.884294 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:01.884300 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:01.885310 (+ 1010us) server_negotiation.cc:407] Connection header received
0810 00:58:01.887395 (+ 2085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:01.887401 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:01.887488 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:01.887615 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:01.889360 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:01.890583 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:01.892160 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:01.892875 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:01.893035 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:01.896780 (+ 3745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:01.896789 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:01.896797 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:01.896843 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:01.899435 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:01.900415 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:01.900419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:01.900422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:01.900541 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:01.901043 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:01.901046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:01.901048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:01.901299 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:01.901452 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:01.902382 (+ 930us) server_negotiation.cc:299] Negotiation successful
0810 00:58:01.902634 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":228,"threads_started":1}
I20260810 00:58:01.904467 25147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:01.884448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:45973 (local address 127.20.81.129:34219)
0810 00:58:01.885144 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:01.885182 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:01.887096 (+ 1914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:01.887902 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:01.887925 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:01.888459 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:01.889187 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:01.889207 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:01.890754 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:01.890758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:01.892012 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:01.892025 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:01.893097 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:01.893101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:01.893296 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:01.894009 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:01.894037 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:01.896554 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:01.899578 (+ 3024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:01.899590 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:01.899611 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:01.900255 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:01.900667 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:01.900675 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:01.900682 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:01.900936 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:01.901445 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:01.901456 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:01.901917 (+ 461us) client_negotiation.cc:769] Sending connection context
0810 00:58:01.902249 (+ 332us) client_negotiation.cc:240] Negotiation successful
0810 00:58:01.902911 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":261,"threads_started":1}
I20260810 00:58:01.906471 25144 heartbeater.cc:347] Connected to a master server at 127.20.81.190:45973
I20260810 00:58:01.906790 25144 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:01.907545 25144 heartbeater.cc:511] Master 127.20.81.190:45973 requested a full tablet report, sending...
I20260810 00:58:01.910029 24927 ts_manager.cc:194] Registered new tserver with Master: afd6e90ae2b8440db791e15f8b7af3c4 (127.20.81.129:33987)
WARNING: no policy specified for kudu/127.20.81.130@KRBTEST.COM; defaulting to no policy
I20260810 00:58:01.912261 24927 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:34219
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
I20260810 00:58:01.942844 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:45973
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:39491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:02.280891 25151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:02.281208 25151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:02.281390 25151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:02.288755 25151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:02.288920 25151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:02.289101 25151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 00:58:02.298960 25151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:39491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:45973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:02.300240 25151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:02.301551 25151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:02.315565 25158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:02.315543 25160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:02.315806 25157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:02.316583 25151 server_base.cc:1034] running on GCE node
I20260810 00:58:02.317230 25151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:02.318221 25151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:02.319509 25151 hybrid_clock.cc:648] HybridClock initialized: now 1786323482319435 us; error 61 us; skew 500 ppm
I20260810 00:58:02.319845 25151 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:02 dist-test-slave-gp6n krb5kdc[24877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323482, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:02.323123 25151 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 00:58:02.324724 25151 webserver.cc:460] Webserver started at http://127.20.81.130:35819/ using document root <none> and password file <none>
I20260810 00:58:02.325433 25151 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:02.325537 25151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:02.325783 25151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:02.329033 25151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "4796aaf674da4dccb111b846e5c221ea"
format_stamp: "Formatted at 2026-08-10 00:58:02 on dist-test-slave-gp6n"
server_key: "9bf0bade643d8218fe8da6400a826cec"
server_key_iv: "ec4d269a7cffb7a304c66a0ce3410455"
server_key_version: "encryptionkey@0"
I20260810 00:58:02.329867 25151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "4796aaf674da4dccb111b846e5c221ea"
format_stamp: "Formatted at 2026-08-10 00:58:02 on dist-test-slave-gp6n"
server_key: "9bf0bade643d8218fe8da6400a826cec"
server_key_iv: "ec4d269a7cffb7a304c66a0ce3410455"
server_key_version: "encryptionkey@0"
I20260810 00:58:02.335178 25151 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 00:58:02.338701 25171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:02.339999 25151 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 00:58:02.340175 25151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "4796aaf674da4dccb111b846e5c221ea"
format_stamp: "Formatted at 2026-08-10 00:58:02 on dist-test-slave-gp6n"
server_key: "9bf0bade643d8218fe8da6400a826cec"
server_key_iv: "ec4d269a7cffb7a304c66a0ce3410455"
server_key_version: "encryptionkey@0"
I20260810 00:58:02.340387 25151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:02.360037 25151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:02.364581 25151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:02.364895 25151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:02.365882 25151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:02.367372 25151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:02.367477 25151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:02.367633 25151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:02.367707 25151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:02.401933 25151 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:36923
I20260810 00:58:02.402024 25302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:36923 every 8 connection(s)
I20260810 00:58:02.403272 25151 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 00:58:02.406045 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 25151
I20260810 00:58:02.406255 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 00:58:02.406924 20806 external_mini_cluster.cc:1428] Setting key b1da90f44e17a832d4a78c6a20a846c6
Aug 10 00:58:02 dist-test-slave-gp6n krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323482, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:58:02.424917 25306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:02.406118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:39189 (local address 127.20.81.190:45973)
0810 00:58:02.406705 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:02.406712 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:02.407813 (+ 1101us) server_negotiation.cc:407] Connection header received
0810 00:58:02.409689 (+ 1876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:02.409695 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:02.409793 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:02.409927 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:02.411475 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:02.412791 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:02.414473 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:02.414911 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:02.415052 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:02.419015 (+ 3963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:02.419020 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:02.419024 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:02.419056 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:02.421772 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:02.422530 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:02.422542 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:02.422546 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:02.422668 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:02.423113 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:02.423118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:02.423121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:02.423404 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:02.423562 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:02.424313 (+ 751us) server_negotiation.cc:299] Negotiation successful
0810 00:58:02.424541 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":294,"threads_started":1}
I20260810 00:58:02.425930 25308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:02.406789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:45973 (local address 127.20.81.130:39189)
0810 00:58:02.407647 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:02.407695 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:02.409413 (+ 1718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:02.410126 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:02.410134 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:02.410728 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:02.411294 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:02.411310 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:02.412978 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:02.412982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:02.414279 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:02.414291 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:02.415146 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:02.415153 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:02.415393 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:02.416073 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:02.416100 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:02.418774 (+ 2674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:02.421918 (+ 3144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:02.421926 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:02.421942 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:02.422363 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:02.422780 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:02.422786 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:02.422789 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:02.422986 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:02.423534 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:02.423542 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:02.423951 (+ 409us) client_negotiation.cc:769] Sending connection context
0810 00:58:02.424212 (+ 261us) client_negotiation.cc:240] Negotiation successful
0810 00:58:02.424734 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":382,"threads_started":1}
I20260810 00:58:02.427613 25304 heartbeater.cc:347] Connected to a master server at 127.20.81.190:45973
I20260810 00:58:02.428005 25304 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:02.428952 25304 heartbeater.cc:511] Master 127.20.81.190:45973 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.81.131@KRBTEST.COM; defaulting to no policy
I20260810 00:58:02.430737 24927 ts_manager.cc:194] Registered new tserver with Master: 4796aaf674da4dccb111b846e5c221ea (127.20.81.130:36923)
I20260810 00:58:02.431593 24927 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:39189
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
I20260810 00:58:02.457743 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:45973
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:39491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:02.782403 25311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:02.782706 25311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:02.782898 25311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:02.790119 25311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:02.790247 25311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:02.790463 25311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 00:58:02.800107 25311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:39491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:45973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:02.801357 25311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:02.802748 25311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:02.816506 25316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:02.816538 25317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:02.816583 25320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:02.817449 25311 server_base.cc:1034] running on GCE node
I20260810 00:58:02.818084 25311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:02.819196 25311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:02.820441 25311 hybrid_clock.cc:648] HybridClock initialized: now 1786323482820400 us; error 33 us; skew 500 ppm
I20260810 00:58:02.820777 25311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:02 dist-test-slave-gp6n krb5kdc[24877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323482, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:02.824215 25311 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 00:58:02.825655 25311 webserver.cc:460] Webserver started at http://127.20.81.131:36769/ using document root <none> and password file <none>
I20260810 00:58:02.826345 25311 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:02.826509 25311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:02.826915 25311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:02.829958 25311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "12dcc9f8085442d4a06b488e3cec06c8"
format_stamp: "Formatted at 2026-08-10 00:58:02 on dist-test-slave-gp6n"
server_key: "fd0c6221787464efda01817390773cbf"
server_key_iv: "6e54745401ead50e85c32b64e1ae6e8c"
server_key_version: "encryptionkey@0"
I20260810 00:58:02.830804 25311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "12dcc9f8085442d4a06b488e3cec06c8"
format_stamp: "Formatted at 2026-08-10 00:58:02 on dist-test-slave-gp6n"
server_key: "fd0c6221787464efda01817390773cbf"
server_key_iv: "6e54745401ead50e85c32b64e1ae6e8c"
server_key_version: "encryptionkey@0"
I20260810 00:58:02.835954 25311 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 00:58:02.839922 25328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:02.841346 25311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 00:58:02.841476 25311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "12dcc9f8085442d4a06b488e3cec06c8"
format_stamp: "Formatted at 2026-08-10 00:58:02 on dist-test-slave-gp6n"
server_key: "fd0c6221787464efda01817390773cbf"
server_key_iv: "6e54745401ead50e85c32b64e1ae6e8c"
server_key_version: "encryptionkey@0"
I20260810 00:58:02.841725 25311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:02.860891 25311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:02.865120 25311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:02.865419 25311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:02.866451 25311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:02.867918 25311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:02.867991 25311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:02.868161 25311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:02.868212 25311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:02.902711 25311 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:40043
I20260810 00:58:02.902760 25456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:40043 every 8 connection(s)
I20260810 00:58:02.904040 25311 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 00:58:02.908313 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 25311
I20260810 00:58:02.908515 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 00:58:02.909126 20806 external_mini_cluster.cc:1428] Setting key d726480b525e4ec5f02bab59ba5d1695
I20260810 00:58:02.915620 25144 heartbeater.cc:503] Master 127.20.81.190:45973 was elected leader, sending a full tablet report...
Aug 10 00:58:02 dist-test-slave-gp6n krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323482, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:58:02.931041 25306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:02.906724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:51255 (local address 127.20.81.190:45973)
0810 00:58:02.906916 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:02.906920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:02.908255 (+ 1335us) server_negotiation.cc:407] Connection header received
0810 00:58:02.910822 (+ 2567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:02.910828 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:02.910922 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:02.911023 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:02.912850 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:02.913629 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:02.915286 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:02.915697 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:02.915864 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:02.925159 (+ 9295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:02.925164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:02.925167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:02.925199 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:02.927951 (+ 2752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:02.928726 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:02.928731 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:02.928733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:02.928869 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:02.929350 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:02.929354 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:02.929356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:02.929679 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:02.929810 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:02.930596 (+ 786us) server_negotiation.cc:299] Negotiation successful
0810 00:58:02.930788 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 00:58:02.932017 25461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:02.907218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:45973 (local address 127.20.81.131:51255)
0810 00:58:02.908052 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:02.908114 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:02.910495 (+ 2381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:02.911285 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:02.911298 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:02.911947 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:02.912671 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:02.912693 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:02.913764 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:02.913768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:02.915153 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:02.915165 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:02.915957 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:02.915965 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:02.916196 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:02.922021 (+ 5825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:02.922051 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:02.924945 (+ 2894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:02.928074 (+ 3129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:02.928081 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:02.928095 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:02.928563 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:02.929027 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:02.929032 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:02.929042 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:02.929226 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:02.929800 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:02.929805 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:02.930150 (+ 345us) client_negotiation.cc:769] Sending connection context
0810 00:58:02.930389 (+ 239us) client_negotiation.cc:240] Negotiation successful
0810 00:58:02.930849 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":292,"threads_started":1}
I20260810 00:58:02.933563 25458 heartbeater.cc:347] Connected to a master server at 127.20.81.190:45973
I20260810 00:58:02.933864 25458 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:02.934651 25458 heartbeater.cc:511] Master 127.20.81.190:45973 requested a full tablet report, sending...
I20260810 00:58:02.936475 24927 ts_manager.cc:194] Registered new tserver with Master: 12dcc9f8085442d4a06b488e3cec06c8 (127.20.81.131:40043)
I20260810 00:58:02.937582 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:58:02.937578 24927 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:51255
Aug 10 00:58:02 dist-test-slave-gp6n krb5kdc[24877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 00:58:02 dist-test-slave-gp6n krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.129@KRBTEST.COM
I20260810 00:58:02.994973 25466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:02.979245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52926 (local address 127.20.81.129:33987)
0810 00:58:02.979883 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:02.979891 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:02.979920 (+ 29us) server_negotiation.cc:407] Connection header received
0810 00:58:02.980032 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:02.980039 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:02.980259 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:02.980427 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:02.981879 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:02.982989 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:02.984237 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:02.984811 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:02.984956 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:02.989180 (+ 4224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:02.989193 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:02.989208 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:02.989246 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:02.992314 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:02.992956 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:02.992963 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:02.992966 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:02.993095 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:02.993511 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:02.993514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:02.993516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:02.993866 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:02.994113 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:02.994381 (+ 268us) server_negotiation.cc:299] Negotiation successful
0810 00:58:02.994662 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":308,"threads_started":1}
Aug 10 00:58:03 dist-test-slave-gp6n krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:58:03.015558 25306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:02.999995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60542 (local address 127.20.81.190:45973)
0810 00:58:03.000229 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:03.000236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:03.000476 (+ 240us) server_negotiation.cc:407] Connection header received
0810 00:58:03.000706 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:03.000710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:03.000788 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:03.000914 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:03.002253 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:03.003152 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:03.004536 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:03.005025 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:03.005174 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:03.009518 (+ 4344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:03.009524 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:03.009527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:03.009566 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:03.012530 (+ 2964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:03.013278 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:03.013286 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:03.013290 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:03.013439 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:03.013930 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:03.013937 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:03.013941 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:03.014295 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:03.014631 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:03.014953 (+ 322us) server_negotiation.cc:299] Negotiation successful
0810 00:58:03.015192 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260810 00:58:03.016142 24927 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60542
I20260810 00:58:03.017261 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 24996
I20260810 00:58:03.031381 25466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:03.018293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52934 (local address 127.20.81.129:33987)
0810 00:58:03.018514 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:03.018518 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:03.018796 (+ 278us) server_negotiation.cc:407] Connection header received
0810 00:58:03.018922 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:03.018926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:03.019027 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:03.019115 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:03.020444 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:03.021252 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:03.022615 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:03.023080 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:03.023192 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:03.025752 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:03.025757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:03.025761 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:03.025835 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:03.028897 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:03.029559 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:03.029563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:03.029566 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:03.029698 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:03.030075 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:03.030079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:03.030081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:03.030367 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:03.030642 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:03.030948 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 00:58:03.031125 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 00:58:03.032070 25139 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:03.121397 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 24996
I20260810 00:58:03.141688 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 25151
Aug 10 00:58:03 dist-test-slave-gp6n krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.130@KRBTEST.COM
I20260810 00:58:03.158967 25471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:03.142772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40478 (local address 127.20.81.130:36923)
0810 00:58:03.143398 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:03.143405 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:03.143431 (+ 26us) server_negotiation.cc:407] Connection header received
0810 00:58:03.143516 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:03.143522 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:03.143732 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:03.143863 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:03.145226 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:03.146448 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:03.147666 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:03.148513 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:03.148658 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:03.153092 (+ 4434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:03.153103 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:03.153117 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:03.153153 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:03.156123 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:03.156759 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:03.156767 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:03.156769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:03.156925 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:03.157389 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:03.157392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:03.157394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:03.157743 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:03.157999 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:03.158267 (+ 268us) server_negotiation.cc:299] Negotiation successful
0810 00:58:03.158544 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":301,"threads_started":1}
I20260810 00:58:03.160370 25298 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:03.243888 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 25151
I20260810 00:58:03.264536 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 25311
Aug 10 00:58:03 dist-test-slave-gp6n krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.131@KRBTEST.COM
I20260810 00:58:03.282230 25474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:03.265705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48498 (local address 127.20.81.131:40043)
0810 00:58:03.266352 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:03.266359 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:03.266389 (+ 30us) server_negotiation.cc:407] Connection header received
0810 00:58:03.266630 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:03.266638 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:03.266857 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:03.266995 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:03.268239 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:03.269326 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:03.270743 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:03.271338 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:03.271440 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:03.275729 (+ 4289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:03.275740 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:03.275754 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:03.275791 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:03.279393 (+ 3602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:03.280032 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:03.280041 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:03.280045 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:03.280195 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:03.280522 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:03.280527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:03.280530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:03.281015 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:03.281317 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:03.281572 (+ 255us) server_negotiation.cc:299] Negotiation successful
0810 00:58:03.281843 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":355,"threads_started":1}
I20260810 00:58:03.283735 25452 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:03.367800 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 25311
I20260810 00:58:03.388060 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 24893
I20260810 00:58:03.389400 24962 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:03.494837 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 24893
2026-08-10T00:58:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4126 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:58:03 dist-test-slave-gp6n krb5kdc[25479](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:58:03 dist-test-slave-gp6n krb5kdc[25479](info): set up 2 sockets
Aug 10 00:58:03 dist-test-slave-gp6n krb5kdc[25479](info): commencing operation
krb5kdc: starting...
W20260810 00:58:04.681536 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 00:58:04 dist-test-slave-gp6n krb5kdc[25479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:58:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:58:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 00:58:04.906237 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:34173
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:33699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:34173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:05.241410 25496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:05.241698 25496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:05.241791 25496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:05.248757 25496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:58:05.248869 25496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:05.248919 25496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:05.248976 25496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:58:05.249023 25496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:58:05.259478 25496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:33699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:34173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:34173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:05.260725 25496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:05.261984 25496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:05.274070 25503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:05.274121 25504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:05.274029 25506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:05.275295 25496 server_base.cc:1034] running on GCE node
I20260810 00:58:05.276039 25496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:05.277423 25496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:05.278611 25496 hybrid_clock.cc:648] HybridClock initialized: now 1786323485278552 us; error 58 us; skew 500 ppm
I20260810 00:58:05.278936 25496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:05 dist-test-slave-gp6n krb5kdc[25479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323485, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:05.282092 25496 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 00:58:05.283439 25496 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:05.283558 25496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:05.283927 25496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:05.287010 25496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "ef62c17307f94be1b389be0c2b8c647a"
format_stamp: "Formatted at 2026-08-10 00:58:05 on dist-test-slave-gp6n"
server_key: "c6ce2758da103828f424b0668cf8d4d8"
server_key_iv: "7f8ad0ba2ea1bb12afb9b52ad14936ee"
server_key_version: "encryptionkey@0"
I20260810 00:58:05.287846 25496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "ef62c17307f94be1b389be0c2b8c647a"
format_stamp: "Formatted at 2026-08-10 00:58:05 on dist-test-slave-gp6n"
server_key: "c6ce2758da103828f424b0668cf8d4d8"
server_key_iv: "7f8ad0ba2ea1bb12afb9b52ad14936ee"
server_key_version: "encryptionkey@0"
I20260810 00:58:05.292965 25496 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 00:58:05.296414 25513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:05.297652 25496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 00:58:05.297837 25496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "ef62c17307f94be1b389be0c2b8c647a"
format_stamp: "Formatted at 2026-08-10 00:58:05 on dist-test-slave-gp6n"
server_key: "c6ce2758da103828f424b0668cf8d4d8"
server_key_iv: "7f8ad0ba2ea1bb12afb9b52ad14936ee"
server_key_version: "encryptionkey@0"
I20260810 00:58:05.298048 25496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:05.318140 25496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:05.322885 25496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:05.323187 25496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:05.343420 25496 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:34173
I20260810 00:58:05.343442 25572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:34173 every 8 connection(s)
I20260810 00:58:05.344455 25496 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:58:05.348331 25573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:05.349617 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 25496
I20260810 00:58:05.349856 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 00:58:05.350778 20806 external_mini_cluster.cc:1428] Setting key ece40d72f03a1202de0e9a4ca6d2fef2
I20260810 00:58:05.363554 25573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a: Bootstrap starting.
Aug 10 00:58:05 dist-test-slave-gp6n krb5kdc[25479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:58:05.367394 25573 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:05.368630 25573 log.cc:824] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:05.371699 25573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a: No bootstrap required, opened a new log
I20260810 00:58:05.375108 25576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:05.353229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41820 (local address 127.20.81.190:34173)
0810 00:58:05.354182 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:05.354197 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:05.354244 (+ 47us) server_negotiation.cc:407] Connection header received
0810 00:58:05.355878 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:05.355907 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:05.356322 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:05.356671 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:05.358580 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:05.359910 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:05.361735 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:05.362395 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:05.362601 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:05.367321 (+ 4720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:05.367329 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:05.367344 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:05.367378 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:05.370994 (+ 3616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:05.371649 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:05.371659 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:05.371671 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:05.371872 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:05.372278 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:05.372283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:05.372286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:05.372831 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:05.373182 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:05.373617 (+ 435us) server_negotiation.cc:299] Negotiation successful
0810 00:58:05.374043 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":722,"thread_start_us":437,"threads_started":1}
I20260810 00:58:05.379496 25573 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef62c17307f94be1b389be0c2b8c647a" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 34173 } }
I20260810 00:58:05.379868 25573 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:05.379976 25573 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef62c17307f94be1b389be0c2b8c647a, State: Initialized, Role: FOLLOWER
I20260810 00:58:05.380568 25573 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef62c17307f94be1b389be0c2b8c647a" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 34173 } }
I20260810 00:58:05.380815 25573 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:58:05.380956 25573 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:58:05.381134 25573 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:05.383551 25573 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef62c17307f94be1b389be0c2b8c647a" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 34173 } }
I20260810 00:58:05.384037 25573 leader_election.cc:302] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef62c17307f94be1b389be0c2b8c647a; no voters:
I20260810 00:58:05.384655 25573 leader_election.cc:288] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:58:05.384788 25578 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:58:05.387581 25573 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:58:05.390622 25578 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a [term 1 LEADER]: Becoming Leader. State: Replica: ef62c17307f94be1b389be0c2b8c647a, State: Running, Role: LEADER
I20260810 00:58:05.391170 25578 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef62c17307f94be1b389be0c2b8c647a" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 34173 } }
I20260810 00:58:05.394447 25580 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef62c17307f94be1b389be0c2b8c647a. Latest consensus state: current_term: 1 leader_uuid: "ef62c17307f94be1b389be0c2b8c647a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef62c17307f94be1b389be0c2b8c647a" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 34173 } } }
I20260810 00:58:05.394811 25580 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:05.394312 25579 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef62c17307f94be1b389be0c2b8c647a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef62c17307f94be1b389be0c2b8c647a" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 34173 } } }
I20260810 00:58:05.395228 25579 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a [sys.catalog]: This master's current role is: LEADER
W20260810 00:58:05.398509 25593 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 00:58:05.398676 25593 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 00:58:05.399814 25594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:58:05.401630 25594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:58:05.410547 25594 catalog_manager.cc:1347] Generated new cluster ID: e9f4330446974aff980d832b16e192a8
I20260810 00:58:05.410645 25594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:58:05.430588 25594 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:58:05.431990 25594 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:58:05.447227 25594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a: Generated new TSK 0
I20260810 00:58:05.448072 25594 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.81.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
I20260810 00:58:05.513758 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:34173
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:33699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:05.853917 25600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:05.854209 25600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:05.854358 25600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:05.861644 25600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:05.861763 25600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:05.861881 25600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 00:58:05.871805 25600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:33699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:34173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:05.873092 25600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:05.874392 25600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:05.887498 25605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:05.887456 25608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:05.887558 25606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:05.888949 25600 server_base.cc:1034] running on GCE node
I20260810 00:58:05.889592 25600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:05.890615 25600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:05.891841 25600 hybrid_clock.cc:648] HybridClock initialized: now 1786323485891803 us; error 31 us; skew 500 ppm
I20260810 00:58:05.892159 25600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:05 dist-test-slave-gp6n krb5kdc[25479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323485, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:05.895584 25600 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 00:58:05.896834 25600 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:05.896931 25600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:05.897253 25600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:05.900480 25600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "7793c766638f49e0aad86f3a203560b3"
format_stamp: "Formatted at 2026-08-10 00:58:05 on dist-test-slave-gp6n"
server_key: "653f80f45c57259d0da197e869df253c"
server_key_iv: "7da8356119038fa297c7870be53dcc38"
server_key_version: "encryptionkey@0"
I20260810 00:58:05.901419 25600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "7793c766638f49e0aad86f3a203560b3"
format_stamp: "Formatted at 2026-08-10 00:58:05 on dist-test-slave-gp6n"
server_key: "653f80f45c57259d0da197e869df253c"
server_key_iv: "7da8356119038fa297c7870be53dcc38"
server_key_version: "encryptionkey@0"
I20260810 00:58:05.906951 25600 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 00:58:05.910665 25614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:05.912056 25600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 00:58:05.912261 25600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "7793c766638f49e0aad86f3a203560b3"
format_stamp: "Formatted at 2026-08-10 00:58:05 on dist-test-slave-gp6n"
server_key: "653f80f45c57259d0da197e869df253c"
server_key_iv: "7da8356119038fa297c7870be53dcc38"
server_key_version: "encryptionkey@0"
I20260810 00:58:05.912541 25600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:05.929540 25600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:05.933768 25600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:05.934067 25600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:05.935119 25600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:05.936434 25600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:05.936558 25600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:05.936691 25600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:05.936782 25600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:05.970757 25600 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:36785
I20260810 00:58:05.970813 25748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:36785 every 8 connection(s)
I20260810 00:58:05.971663 25600 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 00:58:05.976866 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 25600
I20260810 00:58:05.977066 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 00:58:05.977720 20806 external_mini_cluster.cc:1428] Setting key 4f15aade767d0fb7278bbdc243f50f16
Aug 10 00:58:05 dist-test-slave-gp6n krb5kdc[25479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323485, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:58:05.993032 25751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:05.974268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:42495 (local address 127.20.81.190:34173)
0810 00:58:05.974751 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:05.974755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:05.975777 (+ 1022us) server_negotiation.cc:407] Connection header received
0810 00:58:05.977692 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:05.977697 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:05.977794 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:05.977923 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:05.979766 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:05.980509 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:05.982100 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:05.982580 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:05.982721 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:05.986273 (+ 3552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:05.986279 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:05.986282 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:05.986314 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:05.989607 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:05.990616 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:05.990621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:05.990623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:05.990785 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:05.991314 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:05.991318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:05.991320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:05.991585 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:05.991706 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:05.992595 (+ 889us) server_negotiation.cc:299] Negotiation successful
0810 00:58:05.992767 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":259,"threads_started":1}
I20260810 00:58:05.994640 25752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:05.974873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:34173 (local address 127.20.81.129:42495)
0810 00:58:05.975604 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:05.975651 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:05.977417 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:05.978167 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:05.978175 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:05.978829 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:05.979590 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:05.979611 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:05.980654 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:05.980664 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:05.981956 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:05.981968 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:05.982832 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:05.982840 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:05.983072 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:05.983805 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:05.983834 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:05.986047 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:05.989755 (+ 3708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:05.989766 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:05.989787 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:05.990500 (+ 713us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:05.990909 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:05.990916 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:05.990923 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:05.991187 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:05.991723 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:05.991732 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:05.992189 (+ 457us) client_negotiation.cc:769] Sending connection context
0810 00:58:05.992496 (+ 307us) client_negotiation.cc:240] Negotiation successful
0810 00:58:05.993108 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":266,"threads_started":1}
I20260810 00:58:05.996254 25749 heartbeater.cc:347] Connected to a master server at 127.20.81.190:34173
I20260810 00:58:05.996644 25749 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:05.997470 25749 heartbeater.cc:511] Master 127.20.81.190:34173 requested a full tablet report, sending...
I20260810 00:58:05.999769 25533 ts_manager.cc:194] Registered new tserver with Master: 7793c766638f49e0aad86f3a203560b3 (127.20.81.129:36785)
I20260810 00:58:06.001328 25533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:42495
WARNING: no policy specified for kudu/127.20.81.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
I20260810 00:58:06.033494 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:34173
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:33699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:06.370146 25755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:06.370508 25755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:06.370673 25755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:06.377882 25755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:06.378033 25755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:06.378227 25755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 00:58:06.388487 25755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:33699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:34173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:06.389983 25755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:06.391407 25755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:06.404513 25761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:06.404487 25762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:06.404538 25765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:06.405300 25755 server_base.cc:1034] running on GCE node
I20260810 00:58:06.405944 25755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:06.406911 25755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:06.408116 25755 hybrid_clock.cc:648] HybridClock initialized: now 1786323486408078 us; error 25 us; skew 500 ppm
I20260810 00:58:06.408434 25755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:06 dist-test-slave-gp6n krb5kdc[25479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323486, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:06.411847 25755 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 00:58:06.412981 25755 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:06.413069 25755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:06.413326 25755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:06.416271 25755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "90f7cab9bcd9478294adfa2fb0988604"
format_stamp: "Formatted at 2026-08-10 00:58:06 on dist-test-slave-gp6n"
server_key: "ce580d593208875454ef12684d155ee4"
server_key_iv: "bb1f72153141d6fffa266ad63e4e003a"
server_key_version: "encryptionkey@0"
I20260810 00:58:06.417208 25755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "90f7cab9bcd9478294adfa2fb0988604"
format_stamp: "Formatted at 2026-08-10 00:58:06 on dist-test-slave-gp6n"
server_key: "ce580d593208875454ef12684d155ee4"
server_key_iv: "bb1f72153141d6fffa266ad63e4e003a"
server_key_version: "encryptionkey@0"
I20260810 00:58:06.422506 25755 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260810 00:58:06.425913 25772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:06.427271 25755 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:58:06.427438 25755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "90f7cab9bcd9478294adfa2fb0988604"
format_stamp: "Formatted at 2026-08-10 00:58:06 on dist-test-slave-gp6n"
server_key: "ce580d593208875454ef12684d155ee4"
server_key_iv: "bb1f72153141d6fffa266ad63e4e003a"
server_key_version: "encryptionkey@0"
I20260810 00:58:06.427668 25755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:06.461057 25755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:06.465478 25755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:06.465796 25755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:06.466976 25755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:06.468389 25755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:06.468477 25755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:06.468636 25755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:06.468711 25755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:06.505095 25755 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:36425
I20260810 00:58:06.505148 25895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:36425 every 8 connection(s)
I20260810 00:58:06.505998 25755 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 00:58:06.507222 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 25755
I20260810 00:58:06.507428 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 00:58:06.508023 20806 external_mini_cluster.cc:1428] Setting key e47227731822ad7e7ec53842673f74ce
Aug 10 00:58:06 dist-test-slave-gp6n krb5kdc[25479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323486, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:58:06.528100 25898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:06.508922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:55845 (local address 127.20.81.190:34173)
0810 00:58:06.509352 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:06.509357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:06.512483 (+ 3126us) server_negotiation.cc:407] Connection header received
0810 00:58:06.512543 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:06.512546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:06.512604 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:06.512687 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:06.514860 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:06.515559 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:06.517993 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:06.518456 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:06.518554 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:06.522232 (+ 3678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:06.522237 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:06.522240 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:06.522272 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:06.524974 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:06.525694 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:06.525700 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:06.525704 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:06.525828 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:06.526255 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:06.526261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:06.526264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:06.526642 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:06.526819 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:06.527496 (+ 677us) server_negotiation.cc:299] Negotiation successful
0810 00:58:06.527720 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":204,"threads_started":1}
I20260810 00:58:06.529063 25899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:06.509572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:34173 (local address 127.20.81.130:55845)
0810 00:58:06.510440 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:06.510496 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:06.512153 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:06.512989 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:06.513003 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:06.513813 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:06.514665 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:06.514701 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:06.515711 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:06.515719 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:06.517828 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:06.517845 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:06.518672 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:06.518681 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:06.518959 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:06.519647 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:06.519668 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:06.522015 (+ 2347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:06.525113 (+ 3098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:06.525121 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:06.525137 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:06.525571 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:06.525948 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:06.525954 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:06.525957 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:06.526140 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:06.526773 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:06.526781 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:06.527129 (+ 348us) client_negotiation.cc:769] Sending connection context
0810 00:58:06.527409 (+ 280us) client_negotiation.cc:240] Negotiation successful
0810 00:58:06.527928 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":333,"threads_started":1}
I20260810 00:58:06.530651 25896 heartbeater.cc:347] Connected to a master server at 127.20.81.190:34173
I20260810 00:58:06.530967 25896 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:06.531692 25896 heartbeater.cc:511] Master 127.20.81.190:34173 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.81.131@KRBTEST.COM; defaulting to no policy
I20260810 00:58:06.533370 25533 ts_manager.cc:194] Registered new tserver with Master: 90f7cab9bcd9478294adfa2fb0988604 (127.20.81.130:36425)
I20260810 00:58:06.534199 25533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:55845
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
I20260810 00:58:06.561415 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:34173
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:33699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:06.893352 25902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:06.893636 25902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:06.893790 25902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:06.901039 25902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:06.901229 25902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:06.901423 25902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 00:58:06.911628 25902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:33699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:34173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:06.912928 25902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:06.914215 25902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:06.927444 25907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:06.927421 25908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:06.927564 25910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:06.929492 25902 server_base.cc:1034] running on GCE node
I20260810 00:58:06.930058 25902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:06.931087 25902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:06.932327 25902 hybrid_clock.cc:648] HybridClock initialized: now 1786323486932277 us; error 38 us; skew 500 ppm
I20260810 00:58:06.932660 25902 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:06 dist-test-slave-gp6n krb5kdc[25479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323486, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:06.935827 25902 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 00:58:06.937001 25902 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:06.937088 25902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:06.937337 25902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:06.940164 25902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "4ca6e2118da049ab9aa5f9ef64cf66e4"
format_stamp: "Formatted at 2026-08-10 00:58:06 on dist-test-slave-gp6n"
server_key: "c94f83dbae58abb66889b51f5f7d44d7"
server_key_iv: "62a129576fb7a8cd6c24e81974268efe"
server_key_version: "encryptionkey@0"
I20260810 00:58:06.940944 25902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "4ca6e2118da049ab9aa5f9ef64cf66e4"
format_stamp: "Formatted at 2026-08-10 00:58:06 on dist-test-slave-gp6n"
server_key: "c94f83dbae58abb66889b51f5f7d44d7"
server_key_iv: "62a129576fb7a8cd6c24e81974268efe"
server_key_version: "encryptionkey@0"
I20260810 00:58:06.945981 25902 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 00:58:06.949393 25917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:06.951256 25902 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:58:06.951496 25902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "4ca6e2118da049ab9aa5f9ef64cf66e4"
format_stamp: "Formatted at 2026-08-10 00:58:06 on dist-test-slave-gp6n"
server_key: "c94f83dbae58abb66889b51f5f7d44d7"
server_key_iv: "62a129576fb7a8cd6c24e81974268efe"
server_key_version: "encryptionkey@0"
I20260810 00:58:06.951748 25902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:06.972687 25902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:06.977074 25902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:06.977360 25902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:06.978461 25902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:06.979794 25902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:06.979889 25902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:06.980038 25902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:06.980127 25902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:07.004483 25749 heartbeater.cc:503] Master 127.20.81.190:34173 was elected leader, sending a full tablet report...
I20260810 00:58:07.019426 25902 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:44761
I20260810 00:58:07.019523 26043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:44761 every 8 connection(s)
I20260810 00:58:07.020283 25902 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 00:58:07.023644 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 25902
I20260810 00:58:07.023859 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 00:58:07.024447 20806 external_mini_cluster.cc:1428] Setting key e365a9f18472819c42a39f3575576efd
Aug 10 00:58:07 dist-test-slave-gp6n krb5kdc[25479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323486, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:58:07.042522 25898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.023005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:34567 (local address 127.20.81.190:34173)
0810 00:58:07.023203 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:07.023208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:07.024611 (+ 1403us) server_negotiation.cc:407] Connection header received
0810 00:58:07.026941 (+ 2330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:07.026947 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:07.027054 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:07.027174 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:07.028816 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.029638 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.031807 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.032240 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.032394 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:07.036334 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:07.036340 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:07.036343 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:07.036386 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:07.039327 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:07.040129 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:07.040133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:07.040135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:07.040255 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:07.040706 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:07.040710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:07.040712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:07.040988 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:07.041168 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:07.041885 (+ 717us) server_negotiation.cc:299] Negotiation successful
0810 00:58:07.042128 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 00:58:07.043516 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.023444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:34173 (local address 127.20.81.131:34567)
0810 00:58:07.024368 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:07.024410 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:07.026661 (+ 2251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:07.027431 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:07.027439 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:07.027993 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:07.028615 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:07.028634 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.029799 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.029805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:07.031642 (+ 1837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:07.031665 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.032487 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.032494 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:07.032753 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:07.033500 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:07.033525 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:07.036090 (+ 2565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:07.039481 (+ 3391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:07.039491 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:07.039508 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:07.039983 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:07.040370 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:07.040375 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:07.040379 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:07.040574 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:07.041128 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:07.041137 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:07.041500 (+ 363us) client_negotiation.cc:769] Sending connection context
0810 00:58:07.041783 (+ 283us) client_negotiation.cc:240] Negotiation successful
0810 00:58:07.042361 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":423,"threads_started":1}
I20260810 00:58:07.045125 26044 heartbeater.cc:347] Connected to a master server at 127.20.81.190:34173
I20260810 00:58:07.045414 26044 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:07.046054 26044 heartbeater.cc:511] Master 127.20.81.190:34173 requested a full tablet report, sending...
I20260810 00:58:07.047561 25533 ts_manager.cc:194] Registered new tserver with Master: 4ca6e2118da049ab9aa5f9ef64cf66e4 (127.20.81.131:44761)
I20260810 00:58:07.048324 25533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:34567
I20260810 00:58:07.052691 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:58:07.070580 25898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.057555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41828 (local address 127.20.81.190:34173)
0810 00:58:07.057806 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:07.057813 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:07.058172 (+ 359us) server_negotiation.cc:407] Connection header received
0810 00:58:07.058366 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:07.058371 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:07.058501 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:07.058582 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:07.060094 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.060889 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.062158 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.062707 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.062814 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:07.064992 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:07.064996 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:07.065000 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:07.065028 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:07.067782 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:07.068426 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:07.068430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:07.068433 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:07.068600 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:07.068973 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:07.068977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:07.068979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:07.069382 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:07.069595 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:07.069973 (+ 378us) server_negotiation.cc:299] Negotiation successful
0810 00:58:07.070138 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260810 00:58:07.074817 25533 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41828:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:58:07.081533 25533 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:58:07.106150 26057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.094688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49308 (local address 127.20.81.129:36785)
0810 00:58:07.097129 (+ 2441us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:07.097140 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:07.097173 (+ 33us) server_negotiation.cc:407] Connection header received
0810 00:58:07.098121 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:07.098128 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:07.098284 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:07.098506 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:07.099332 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.100878 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.103604 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.104538 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.104697 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:07.104950 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:07.105569 (+ 619us) server_negotiation.cc:299] Negotiation successful
0810 00:58:07.105761 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2267,"thread_start_us":2117,"threads_started":1}
I20260810 00:58:07.107152 26056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.094447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:36785 (local address 127.0.0.1:49308)
0810 00:58:07.095424 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:07.095489 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:07.095714 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:07.098490 (+ 2776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:07.098494 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:07.098531 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:07.099102 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:07.099122 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.101005 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.101037 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:07.103432 (+ 2395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:07.103454 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.105099 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.105107 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:07.105302 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:07.105352 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 00:58:07.105495 (+ 143us) client_negotiation.cc:240] Negotiation successful
0810 00:58:07.105697 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":437,"threads_started":1}
I20260810 00:58:07.107858 26058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.094999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38440 (local address 127.20.81.131:44761)
0810 00:58:07.096619 (+ 1620us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:07.096631 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:07.097231 (+ 600us) server_negotiation.cc:407] Connection header received
0810 00:58:07.097424 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:07.097435 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:07.097715 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:07.097908 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:07.099072 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.101144 (+ 2072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.104697 (+ 3553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.106155 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.106279 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:07.107307 (+ 1028us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:07.107398 (+ 91us) server_negotiation.cc:299] Negotiation successful
0810 00:58:07.107575 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1372,"thread_start_us":1459,"threads_started":1}
I20260810 00:58:07.107896 26060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.096340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:36425 (local address 127.0.0.1:48628)
0810 00:58:07.099263 (+ 2923us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:07.099291 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:07.099398 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:07.100048 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:07.100053 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:07.100078 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:07.100396 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:07.100408 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.102497 (+ 2089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.102503 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:07.104910 (+ 2407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:07.104922 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.107021 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.107026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:07.107158 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:07.107181 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 00:58:07.107329 (+ 148us) client_negotiation.cc:240] Negotiation successful
0810 00:58:07.107459 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2778,"mutex_wait_us":54,"thread_start_us":2560,"threads_started":1}
I20260810 00:58:07.108636 26059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.095468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:44761 (local address 127.0.0.1:38440)
0810 00:58:07.097122 (+ 1654us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:07.097149 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:07.097285 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:07.097980 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:07.097992 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:07.098053 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:07.098685 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:07.098704 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.101374 (+ 2670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.101380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:07.104514 (+ 3134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:07.104530 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.106559 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.106565 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:07.106738 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:07.106771 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 00:58:07.106846 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 00:58:07.106990 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"mutex_wait_us":1,"thread_start_us":471,"threads_started":1}
I20260810 00:58:07.108757 26061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.096693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48628 (local address 127.20.81.130:36425)
0810 00:58:07.097790 (+ 1097us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:07.097800 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:07.099380 (+ 1580us) server_negotiation.cc:407] Connection header received
0810 00:58:07.099577 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:07.099585 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:07.099830 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:07.100048 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:07.100590 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.102277 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.105074 (+ 2797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.106537 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.106698 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:07.106879 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:07.108204 (+ 1325us) server_negotiation.cc:299] Negotiation successful
0810 00:58:07.108399 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":945,"thread_start_us":778,"threads_started":1}
I20260810 00:58:07.111610 25676 tablet_service.cc:1467] Processing CreateTablet for tablet a802b224c5f247bebbc93a86cfda1512 (DEFAULT_TABLE table=test-table [id=f648146a37814df5a0fbb06a6cbd75c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:07.112079 25826 tablet_service.cc:1467] Processing CreateTablet for tablet a802b224c5f247bebbc93a86cfda1512 (DEFAULT_TABLE table=test-table [id=f648146a37814df5a0fbb06a6cbd75c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:07.112470 25970 tablet_service.cc:1467] Processing CreateTablet for tablet a802b224c5f247bebbc93a86cfda1512 (DEFAULT_TABLE table=test-table [id=f648146a37814df5a0fbb06a6cbd75c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:07.113234 25676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a802b224c5f247bebbc93a86cfda1512. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:07.113415 25826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a802b224c5f247bebbc93a86cfda1512. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:07.113754 25970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a802b224c5f247bebbc93a86cfda1512. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:07.126783 26062 tablet_bootstrap.cc:492] T a802b224c5f247bebbc93a86cfda1512 P 90f7cab9bcd9478294adfa2fb0988604: Bootstrap starting.
I20260810 00:58:07.131255 26062 tablet_bootstrap.cc:654] T a802b224c5f247bebbc93a86cfda1512 P 90f7cab9bcd9478294adfa2fb0988604: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:07.131282 26064 tablet_bootstrap.cc:492] T a802b224c5f247bebbc93a86cfda1512 P 4ca6e2118da049ab9aa5f9ef64cf66e4: Bootstrap starting.
I20260810 00:58:07.132375 26063 tablet_bootstrap.cc:492] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3: Bootstrap starting.
I20260810 00:58:07.133158 26062 log.cc:824] T a802b224c5f247bebbc93a86cfda1512 P 90f7cab9bcd9478294adfa2fb0988604: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:07.135466 26064 tablet_bootstrap.cc:654] T a802b224c5f247bebbc93a86cfda1512 P 4ca6e2118da049ab9aa5f9ef64cf66e4: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:07.136060 26063 tablet_bootstrap.cc:654] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:07.136540 26064 log.cc:824] T a802b224c5f247bebbc93a86cfda1512 P 4ca6e2118da049ab9aa5f9ef64cf66e4: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:07.136814 26062 tablet_bootstrap.cc:492] T a802b224c5f247bebbc93a86cfda1512 P 90f7cab9bcd9478294adfa2fb0988604: No bootstrap required, opened a new log
I20260810 00:58:07.137120 26062 ts_tablet_manager.cc:1397] T a802b224c5f247bebbc93a86cfda1512 P 90f7cab9bcd9478294adfa2fb0988604: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260810 00:58:07.137403 26063 log.cc:824] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:07.143479 26063 tablet_bootstrap.cc:492] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3: No bootstrap required, opened a new log
I20260810 00:58:07.143790 26063 ts_tablet_manager.cc:1397] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260810 00:58:07.143886 26064 tablet_bootstrap.cc:492] T a802b224c5f247bebbc93a86cfda1512 P 4ca6e2118da049ab9aa5f9ef64cf66e4: No bootstrap required, opened a new log
I20260810 00:58:07.144187 26064 ts_tablet_manager.cc:1397] T a802b224c5f247bebbc93a86cfda1512 P 4ca6e2118da049ab9aa5f9ef64cf66e4: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260810 00:58:07.149791 26062 raft_consensus.cc:348] T a802b224c5f247bebbc93a86cfda1512 P 90f7cab9bcd9478294adfa2fb0988604 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793c766638f49e0aad86f3a203560b3" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 36785 } } peers { permanent_uuid: "4ca6e2118da049ab9aa5f9ef64cf66e4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 44761 } } peers { permanent_uuid: "90f7cab9bcd9478294adfa2fb0988604" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36425 } }
I20260810 00:58:07.150339 26062 raft_consensus.cc:374] T a802b224c5f247bebbc93a86cfda1512 P 90f7cab9bcd9478294adfa2fb0988604 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:07.150507 26062 raft_consensus.cc:729] T a802b224c5f247bebbc93a86cfda1512 P 90f7cab9bcd9478294adfa2fb0988604 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f7cab9bcd9478294adfa2fb0988604, State: Initialized, Role: FOLLOWER
I20260810 00:58:07.151207 26062 consensus_queue.cc:260] T a802b224c5f247bebbc93a86cfda1512 P 90f7cab9bcd9478294adfa2fb0988604 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793c766638f49e0aad86f3a203560b3" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 36785 } } peers { permanent_uuid: "4ca6e2118da049ab9aa5f9ef64cf66e4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 44761 } } peers { permanent_uuid: "90f7cab9bcd9478294adfa2fb0988604" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36425 } }
I20260810 00:58:07.152616 26063 raft_consensus.cc:348] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793c766638f49e0aad86f3a203560b3" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 36785 } } peers { permanent_uuid: "4ca6e2118da049ab9aa5f9ef64cf66e4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 44761 } } peers { permanent_uuid: "90f7cab9bcd9478294adfa2fb0988604" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36425 } }
I20260810 00:58:07.152987 26063 raft_consensus.cc:374] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:07.153105 26063 raft_consensus.cc:729] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793c766638f49e0aad86f3a203560b3, State: Initialized, Role: FOLLOWER
I20260810 00:58:07.153807 26063 consensus_queue.cc:260] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793c766638f49e0aad86f3a203560b3" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 36785 } } peers { permanent_uuid: "4ca6e2118da049ab9aa5f9ef64cf66e4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 44761 } } peers { permanent_uuid: "90f7cab9bcd9478294adfa2fb0988604" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36425 } }
I20260810 00:58:07.155674 25896 heartbeater.cc:503] Master 127.20.81.190:34173 was elected leader, sending a full tablet report...
I20260810 00:58:07.155881 26063 ts_tablet_manager.cc:1428] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3: Time spent starting tablet: real 0.012s user 0.009s sys 0.003s
I20260810 00:58:07.156927 26062 ts_tablet_manager.cc:1428] T a802b224c5f247bebbc93a86cfda1512 P 90f7cab9bcd9478294adfa2fb0988604: Time spent starting tablet: real 0.020s user 0.013s sys 0.004s
I20260810 00:58:07.156594 26064 raft_consensus.cc:348] T a802b224c5f247bebbc93a86cfda1512 P 4ca6e2118da049ab9aa5f9ef64cf66e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793c766638f49e0aad86f3a203560b3" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 36785 } } peers { permanent_uuid: "4ca6e2118da049ab9aa5f9ef64cf66e4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 44761 } } peers { permanent_uuid: "90f7cab9bcd9478294adfa2fb0988604" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36425 } }
I20260810 00:58:07.157081 26064 raft_consensus.cc:374] T a802b224c5f247bebbc93a86cfda1512 P 4ca6e2118da049ab9aa5f9ef64cf66e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:07.157218 26064 raft_consensus.cc:729] T a802b224c5f247bebbc93a86cfda1512 P 4ca6e2118da049ab9aa5f9ef64cf66e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ca6e2118da049ab9aa5f9ef64cf66e4, State: Initialized, Role: FOLLOWER
I20260810 00:58:07.157886 26064 consensus_queue.cc:260] T a802b224c5f247bebbc93a86cfda1512 P 4ca6e2118da049ab9aa5f9ef64cf66e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793c766638f49e0aad86f3a203560b3" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 36785 } } peers { permanent_uuid: "4ca6e2118da049ab9aa5f9ef64cf66e4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 44761 } } peers { permanent_uuid: "90f7cab9bcd9478294adfa2fb0988604" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36425 } }
I20260810 00:58:07.159083 26044 heartbeater.cc:503] Master 127.20.81.190:34173 was elected leader, sending a full tablet report...
I20260810 00:58:07.159790 26064 ts_tablet_manager.cc:1428] T a802b224c5f247bebbc93a86cfda1512 P 4ca6e2118da049ab9aa5f9ef64cf66e4: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
W20260810 00:58:07.223898 25750 tablet.cc:2369] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:58:07.258309 25897 tablet.cc:2369] T a802b224c5f247bebbc93a86cfda1512 P 90f7cab9bcd9478294adfa2fb0988604: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:58:07.269553 26073 raft_consensus.cc:482] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:58:07.269877 26073 raft_consensus.cc:504] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793c766638f49e0aad86f3a203560b3" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 36785 } } peers { permanent_uuid: "4ca6e2118da049ab9aa5f9ef64cf66e4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 44761 } } peers { permanent_uuid: "90f7cab9bcd9478294adfa2fb0988604" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36425 } }
I20260810 00:58:07.271660 26073 leader_election.cc:288] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ca6e2118da049ab9aa5f9ef64cf66e4 (127.20.81.131:44761), 90f7cab9bcd9478294adfa2fb0988604 (127.20.81.130:36425)
W20260810 00:58:07.271883 26045 tablet.cc:2369] T a802b224c5f247bebbc93a86cfda1512 P 4ca6e2118da049ab9aa5f9ef64cf66e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:58:07.280812 26058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.272538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:52013 (local address 127.20.81.131:44761)
0810 00:58:07.272791 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:07.272799 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:07.273349 (+ 550us) server_negotiation.cc:407] Connection header received
0810 00:58:07.273580 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:07.273590 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:07.273705 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:07.273829 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:07.274589 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.275804 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.278376 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.279710 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.279913 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:07.279995 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:07.280396 (+ 401us) server_negotiation.cc:299] Negotiation successful
0810 00:58:07.280531 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 00:58:07.280800 26061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.273503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:47705 (local address 127.20.81.130:36425)
0810 00:58:07.273719 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:07.273727 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:07.274197 (+ 470us) server_negotiation.cc:407] Connection header received
0810 00:58:07.274392 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:07.274444 (+ 52us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:07.274551 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:07.274656 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:07.275135 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.276171 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.278604 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.279842 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.279990 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:07.280065 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:07.280427 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 00:58:07.280552 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 00:58:07.280965 26075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.272370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:44761 (local address 127.20.81.129:52013)
0810 00:58:07.273191 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:07.273209 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:07.273404 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:07.273895 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:07.273907 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:07.273935 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:07.274385 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:07.274395 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.275947 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.275951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:07.278194 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:07.278208 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.279955 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.279965 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:07.280173 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:07.280235 (+ 62us) client_negotiation.cc:769] Sending connection context
0810 00:58:07.280337 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 00:58:07.280477 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"mutex_wait_us":1,"thread_start_us":526,"threads_started":1}
I20260810 00:58:07.281339 26076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.273340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:36425 (local address 127.20.81.129:47705)
0810 00:58:07.274074 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:07.274095 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:07.274224 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:07.274680 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:07.274683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:07.274704 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:07.274961 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:07.274968 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.276302 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.276307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:07.278451 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:07.278467 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.280035 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.280042 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:07.280262 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:07.280303 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 00:58:07.280387 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 00:58:07.280485 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":50,"thread_start_us":377,"threads_started":1}
I20260810 00:58:07.282020 25991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a802b224c5f247bebbc93a86cfda1512" candidate_uuid: "7793c766638f49e0aad86f3a203560b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ca6e2118da049ab9aa5f9ef64cf66e4" is_pre_election: true
I20260810 00:58:07.282666 25991 raft_consensus.cc:2454] T a802b224c5f247bebbc93a86cfda1512 P 4ca6e2118da049ab9aa5f9ef64cf66e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793c766638f49e0aad86f3a203560b3 in term 0.
I20260810 00:58:07.282526 25847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a802b224c5f247bebbc93a86cfda1512" candidate_uuid: "7793c766638f49e0aad86f3a203560b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f7cab9bcd9478294adfa2fb0988604" is_pre_election: true
I20260810 00:58:07.283068 25847 raft_consensus.cc:2454] T a802b224c5f247bebbc93a86cfda1512 P 90f7cab9bcd9478294adfa2fb0988604 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793c766638f49e0aad86f3a203560b3 in term 0.
I20260810 00:58:07.283396 25621 leader_election.cc:302] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ca6e2118da049ab9aa5f9ef64cf66e4, 7793c766638f49e0aad86f3a203560b3; no voters:
I20260810 00:58:07.283823 26073 raft_consensus.cc:2781] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:58:07.283973 26073 raft_consensus.cc:482] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:58:07.284137 26073 raft_consensus.cc:3037] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:07.286459 26073 raft_consensus.cc:504] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793c766638f49e0aad86f3a203560b3" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 36785 } } peers { permanent_uuid: "4ca6e2118da049ab9aa5f9ef64cf66e4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 44761 } } peers { permanent_uuid: "90f7cab9bcd9478294adfa2fb0988604" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36425 } }
I20260810 00:58:07.287058 26073 leader_election.cc:288] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3 [CANDIDATE]: Term 1 election: Requested vote from peers 4ca6e2118da049ab9aa5f9ef64cf66e4 (127.20.81.131:44761), 90f7cab9bcd9478294adfa2fb0988604 (127.20.81.130:36425)
I20260810 00:58:07.287607 25991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a802b224c5f247bebbc93a86cfda1512" candidate_uuid: "7793c766638f49e0aad86f3a203560b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ca6e2118da049ab9aa5f9ef64cf66e4"
I20260810 00:58:07.287716 25847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a802b224c5f247bebbc93a86cfda1512" candidate_uuid: "7793c766638f49e0aad86f3a203560b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f7cab9bcd9478294adfa2fb0988604"
I20260810 00:58:07.287845 25991 raft_consensus.cc:3037] T a802b224c5f247bebbc93a86cfda1512 P 4ca6e2118da049ab9aa5f9ef64cf66e4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:07.287977 25847 raft_consensus.cc:3037] T a802b224c5f247bebbc93a86cfda1512 P 90f7cab9bcd9478294adfa2fb0988604 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:07.290882 25991 raft_consensus.cc:2454] T a802b224c5f247bebbc93a86cfda1512 P 4ca6e2118da049ab9aa5f9ef64cf66e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793c766638f49e0aad86f3a203560b3 in term 1.
I20260810 00:58:07.291158 25847 raft_consensus.cc:2454] T a802b224c5f247bebbc93a86cfda1512 P 90f7cab9bcd9478294adfa2fb0988604 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793c766638f49e0aad86f3a203560b3 in term 1.
I20260810 00:58:07.291245 25621 leader_election.cc:302] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ca6e2118da049ab9aa5f9ef64cf66e4, 7793c766638f49e0aad86f3a203560b3; no voters:
I20260810 00:58:07.291594 26073 raft_consensus.cc:2781] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:58:07.292057 26073 raft_consensus.cc:686] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3 [term 1 LEADER]: Becoming Leader. State: Replica: 7793c766638f49e0aad86f3a203560b3, State: Running, Role: LEADER
I20260810 00:58:07.292562 26073 consensus_queue.cc:237] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793c766638f49e0aad86f3a203560b3" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 36785 } } peers { permanent_uuid: "4ca6e2118da049ab9aa5f9ef64cf66e4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 44761 } } peers { permanent_uuid: "90f7cab9bcd9478294adfa2fb0988604" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36425 } }
I20260810 00:58:07.297528 25531 catalog_manager.cc:5495] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793c766638f49e0aad86f3a203560b3 (127.20.81.129). New cstate: current_term: 1 leader_uuid: "7793c766638f49e0aad86f3a203560b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793c766638f49e0aad86f3a203560b3" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 36785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ca6e2118da049ab9aa5f9ef64cf66e4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 44761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f7cab9bcd9478294adfa2fb0988604" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36425 } health_report { overall_health: UNKNOWN } } }
I20260810 00:58:07.351267 26057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.344713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49324 (local address 127.20.81.129:36785)
0810 00:58:07.344958 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:07.344964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:07.345217 (+ 253us) server_negotiation.cc:407] Connection header received
0810 00:58:07.345411 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:07.345415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:07.345510 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:07.345642 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:58:07.346099 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.347241 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.348641 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.349264 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.349400 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:07.349943 (+ 543us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:58:07.350667 (+ 724us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:58:07.350800 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:07.350923 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 00:58:07.351004 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 00:58:07.360694 25847 raft_consensus.cc:1261] T a802b224c5f247bebbc93a86cfda1512 P 90f7cab9bcd9478294adfa2fb0988604 [term 1 FOLLOWER]: Refusing update from remote peer 7793c766638f49e0aad86f3a203560b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:58:07.360790 25991 raft_consensus.cc:1261] T a802b224c5f247bebbc93a86cfda1512 P 4ca6e2118da049ab9aa5f9ef64cf66e4 [term 1 FOLLOWER]: Refusing update from remote peer 7793c766638f49e0aad86f3a203560b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:58:07.361737 26077 consensus_queue.cc:1035] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ca6e2118da049ab9aa5f9ef64cf66e4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 44761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:58:07.362183 26073 consensus_queue.cc:1035] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f7cab9bcd9478294adfa2fb0988604" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:58:07.388908 25531 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:58:07.389220 25531 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:58:07.393041 25531 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ef62c17307f94be1b389be0c2b8c647a: Sending DeleteTablet for 3 replicas of tablet a802b224c5f247bebbc93a86cfda1512
I20260810 00:58:07.394222 25676 tablet_service.cc:1514] Processing DeleteTablet for tablet a802b224c5f247bebbc93a86cfda1512 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:07 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:49308
I20260810 00:58:07.394335 25970 tablet_service.cc:1514] Processing DeleteTablet for tablet a802b224c5f247bebbc93a86cfda1512 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:07 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:38440
I20260810 00:58:07.394641 25826 tablet_service.cc:1514] Processing DeleteTablet for tablet a802b224c5f247bebbc93a86cfda1512 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:07 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:48628
I20260810 00:58:07.395798 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 25600
I20260810 00:58:07.407078 26092 tablet_replica.cc:330] T a802b224c5f247bebbc93a86cfda1512 P 90f7cab9bcd9478294adfa2fb0988604: stopping tablet replica
I20260810 00:58:07.407771 26092 raft_consensus.cc:2229] T a802b224c5f247bebbc93a86cfda1512 P 90f7cab9bcd9478294adfa2fb0988604 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:58:07.408540 26092 raft_consensus.cc:2258] T a802b224c5f247bebbc93a86cfda1512 P 90f7cab9bcd9478294adfa2fb0988604 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:07.411314 26092 ts_tablet_manager.cc:1905] T a802b224c5f247bebbc93a86cfda1512 P 90f7cab9bcd9478294adfa2fb0988604: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:07.415879 26090 tablet_replica.cc:330] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3: stopping tablet replica
I20260810 00:58:07.416317 26090 raft_consensus.cc:2229] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:58:07.417090 26090 raft_consensus.cc:2258] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:07.417903 26091 tablet_replica.cc:330] T a802b224c5f247bebbc93a86cfda1512 P 4ca6e2118da049ab9aa5f9ef64cf66e4: stopping tablet replica
I20260810 00:58:07.418344 26091 raft_consensus.cc:2229] T a802b224c5f247bebbc93a86cfda1512 P 4ca6e2118da049ab9aa5f9ef64cf66e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:58:07.418921 26091 raft_consensus.cc:2258] T a802b224c5f247bebbc93a86cfda1512 P 4ca6e2118da049ab9aa5f9ef64cf66e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:07.419831 26092 ts_tablet_manager.cc:1918] T a802b224c5f247bebbc93a86cfda1512 P 90f7cab9bcd9478294adfa2fb0988604: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:58:07.419981 26092 log.cc:1196] T a802b224c5f247bebbc93a86cfda1512 P 90f7cab9bcd9478294adfa2fb0988604: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/a802b224c5f247bebbc93a86cfda1512
Aug 10 00:58:07 dist-test-slave-gp6n krb5kdc[25479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.129@KRBTEST.COM
I20260810 00:58:07.420553 26092 ts_tablet_manager.cc:1939] T a802b224c5f247bebbc93a86cfda1512 P 90f7cab9bcd9478294adfa2fb0988604: Deleting consensus metadata
I20260810 00:58:07.420926 26091 ts_tablet_manager.cc:1905] T a802b224c5f247bebbc93a86cfda1512 P 4ca6e2118da049ab9aa5f9ef64cf66e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:07.422297 25517 catalog_manager.cc:4841] TS 90f7cab9bcd9478294adfa2fb0988604 (127.20.81.130:36425): tablet a802b224c5f247bebbc93a86cfda1512 (table test-table [id=f648146a37814df5a0fbb06a6cbd75c5]) successfully deleted
I20260810 00:58:07.426878 26091 ts_tablet_manager.cc:1918] T a802b224c5f247bebbc93a86cfda1512 P 4ca6e2118da049ab9aa5f9ef64cf66e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:58:07.427057 26091 log.cc:1196] T a802b224c5f247bebbc93a86cfda1512 P 4ca6e2118da049ab9aa5f9ef64cf66e4: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/a802b224c5f247bebbc93a86cfda1512
I20260810 00:58:07.427613 26091 ts_tablet_manager.cc:1939] T a802b224c5f247bebbc93a86cfda1512 P 4ca6e2118da049ab9aa5f9ef64cf66e4: Deleting consensus metadata
I20260810 00:58:07.428053 26090 ts_tablet_manager.cc:1905] T a802b224c5f247bebbc93a86cfda1512 P 7793c766638f49e0aad86f3a203560b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:07.428272 26057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.397429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49330 (local address 127.20.81.129:36785)
0810 00:58:07.397649 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:07.397659 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:07.408131 (+ 10472us) server_negotiation.cc:407] Connection header received
0810 00:58:07.408954 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:07.408963 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:07.409050 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:07.409131 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:07.411313 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.412551 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.414468 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.415437 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.415579 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:07.421463 (+ 5884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:07.421477 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:07.421493 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:07.421545 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:07.424546 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:07.425356 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:07.425362 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:07.425365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:07.425506 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:07.425906 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:07.425909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:07.425911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:07.426261 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:07.426577 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:07.427508 (+ 931us) server_negotiation.cc:299] Negotiation successful
0810 00:58:07.427812 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 00:58:07.429410 25517 catalog_manager.cc:4841] TS 4ca6e2118da049ab9aa5f9ef64cf66e4 (127.20.81.131:44761): tablet a802b224c5f247bebbc93a86cfda1512 (table test-table [id=f648146a37814df5a0fbb06a6cbd75c5]) successfully deleted
I20260810 00:58:07.429837 25744 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:07.532601 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 25600
I20260810 00:58:07.554988 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 25755
W20260810 00:58:07.554970 25516 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.81.129:36785 (error 108)
I20260810 00:58:07.557094 26060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.556593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:36785 (local address 127.0.0.1:49338)
0810 00:58:07.556784 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:07.556913 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260810 00:58:07.557387 25516 catalog_manager.cc:4584] TS 7793c766638f49e0aad86f3a203560b3 (127.20.81.129:36785): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a802b224c5f247bebbc93a86cfda1512: Network error: Client connection negotiation failed: client connection to 127.20.81.129:36785: connect: Connection refused (error 111)
Aug 10 00:58:07 dist-test-slave-gp6n krb5kdc[25479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.130@KRBTEST.COM
I20260810 00:58:07.570706 26061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.556000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48644 (local address 127.20.81.130:36425)
0810 00:58:07.556204 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:07.556209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:07.556228 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:58:07.556421 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:07.556431 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:07.556524 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:07.556651 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:07.558135 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.559082 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.560319 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.560761 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.560873 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:07.565014 (+ 4141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:07.565027 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:07.565043 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:07.565075 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:07.568097 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:07.568706 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:07.568713 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:07.568715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:07.568864 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:07.569211 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:07.569214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:07.569216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:07.569589 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:07.569846 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:07.570130 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 00:58:07.570363 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 00:58:07.571421 25891 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:07.611158 26060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.610699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:36785 (local address 127.0.0.1:49350)
0810 00:58:07.610906 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:07.611013 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260810 00:58:07.612226 26060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.611867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:36785 (local address 127.0.0.1:49356)
0810 00:58:07.612016 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:07.612110 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 00:58:07.661816 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 25755
I20260810 00:58:07.672859 26060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.672251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:36785 (local address 127.0.0.1:49368)
0810 00:58:07.672498 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:07.672598 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260810 00:58:07.673934 26060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.673559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:36785 (local address 127.0.0.1:49384)
0810 00:58:07.673716 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:07.673798 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 00:58:07.682125 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 25902
Aug 10 00:58:07 dist-test-slave-gp6n krb5kdc[25479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.131@KRBTEST.COM
I20260810 00:58:07.697547 26058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.682944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38444 (local address 127.20.81.131:44761)
0810 00:58:07.683107 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:07.683111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:07.683130 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:58:07.683282 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:07.683286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:07.683354 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:07.683441 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:07.684721 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.685476 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.686900 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:07.687281 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:07.687450 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:07.692029 (+ 4579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:07.692044 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:07.692060 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:07.692098 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:07.695148 (+ 3050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:07.695755 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:07.695762 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:07.695764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:07.695911 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:07.696216 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:07.696219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:07.696221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:07.696566 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:07.696780 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:07.697016 (+ 236us) server_negotiation.cc:299] Negotiation successful
0810 00:58:07.697172 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 00:58:07.698154 26038 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:07.754688 26060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.754075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:36785 (local address 127.0.0.1:49398)
0810 00:58:07.754324 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:07.754500 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260810 00:58:07.755750 26060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:07.755364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:36785 (local address 127.0.0.1:49406)
0810 00:58:07.755572 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:07.755641 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260810 00:58:07.789746 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 25902
I20260810 00:58:07.809947 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 25496
I20260810 00:58:07.811403 25567 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:07.921541 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 25496
2026-08-10T00:58:07Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4428 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T00:58:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:58:07Z Disabled control of system clock
I20260810 00:58:08.000718 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:35673
--webserver_interface=127.20.81.190
--webserver_port=0
--builtin_ntp_servers=127.20.81.148:41073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:35673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 00:58:08.340698 26103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:08.341143 26103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:08.341287 26103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:08.348713 26103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:58:08.348815 26103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:08.348902 26103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:08.348990 26103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:58:08.349051 26103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:58:08.358042 26103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:41073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:35673
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:35673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:08.359401 26103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:08.361274 26103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:08.373610 26112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:08.373700 26110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:08.373675 26109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:08.374706 26103 server_base.cc:1034] running on GCE node
I20260810 00:58:08.375545 26103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:08.376971 26103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:08.378201 26103 hybrid_clock.cc:648] HybridClock initialized: now 1786323488378166 us; error 53 us; skew 500 ppm
I20260810 00:58:08.378615 26103 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:58:08.385154 26103 webserver.cc:460] Webserver started at http://127.20.81.190:38525/ using document root <none> and password file <none>
I20260810 00:58:08.385854 26103 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:08.385993 26103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:08.386322 26103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:08.389230 26103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "aae752b28b11458688b0565b6cf704cc"
format_stamp: "Formatted at 2026-08-10 00:58:08 on dist-test-slave-gp6n"
server_key: "e02b5e525d9e2868634177c2eb952bf8"
server_key_iv: "60b27bc6f3e129f965890ca29e207bdd"
server_key_version: "encryptionkey@0"
I20260810 00:58:08.390043 26103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "aae752b28b11458688b0565b6cf704cc"
format_stamp: "Formatted at 2026-08-10 00:58:08 on dist-test-slave-gp6n"
server_key: "e02b5e525d9e2868634177c2eb952bf8"
server_key_iv: "60b27bc6f3e129f965890ca29e207bdd"
server_key_version: "encryptionkey@0"
I20260810 00:58:08.395012 26103 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260810 00:58:08.398396 26118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:08.399899 26103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 00:58:08.400094 26103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "aae752b28b11458688b0565b6cf704cc"
format_stamp: "Formatted at 2026-08-10 00:58:08 on dist-test-slave-gp6n"
server_key: "e02b5e525d9e2868634177c2eb952bf8"
server_key_iv: "60b27bc6f3e129f965890ca29e207bdd"
server_key_version: "encryptionkey@0"
I20260810 00:58:08.400318 26103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:08.406256 26103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:08.407111 26103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:08.407431 26103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:08.426215 26103 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:35673
I20260810 00:58:08.426273 26177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:35673 every 8 connection(s)
I20260810 00:58:08.427636 26103 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:58:08.431545 26178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:08.431809 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 26103
I20260810 00:58:08.432026 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 00:58:08.432708 20806 external_mini_cluster.cc:1428] Setting key ca01747877b40242496b5de8c1bf01d2
I20260810 00:58:08.442421 26183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:08.434884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60798 (local address 127.20.81.190:35673)
0810 00:58:08.436032 (+ 1148us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:08.436047 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:08.436101 (+ 54us) server_negotiation.cc:407] Connection header received
0810 00:58:08.438345 (+ 2244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:08.438373 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:08.438966 (+ 593us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:08.439439 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:08.440421 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:08.440429 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:08.440444 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:58:08.440462 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:08.440536 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:58:08.440557 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:08.440777 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:08.440916 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 00:58:08.441180 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":361,"threads_started":1}
I20260810 00:58:08.446322 26178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc: Bootstrap starting.
I20260810 00:58:08.449397 26178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:08.450444 26178 log.cc:824] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:08.453372 26178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc: No bootstrap required, opened a new log
I20260810 00:58:08.460734 26178 raft_consensus.cc:348] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aae752b28b11458688b0565b6cf704cc" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 35673 } }
I20260810 00:58:08.461055 26178 raft_consensus.cc:374] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:08.461126 26178 raft_consensus.cc:729] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aae752b28b11458688b0565b6cf704cc, State: Initialized, Role: FOLLOWER
I20260810 00:58:08.461679 26178 consensus_queue.cc:260] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aae752b28b11458688b0565b6cf704cc" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 35673 } }
I20260810 00:58:08.461922 26178 raft_consensus.cc:388] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:58:08.462122 26178 raft_consensus.cc:482] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:58:08.462306 26178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:08.464419 26178 raft_consensus.cc:504] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aae752b28b11458688b0565b6cf704cc" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 35673 } }
I20260810 00:58:08.464910 26178 leader_election.cc:302] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aae752b28b11458688b0565b6cf704cc; no voters:
I20260810 00:58:08.465536 26178 leader_election.cc:288] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:58:08.465628 26187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:58:08.466382 26187 raft_consensus.cc:686] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc [term 1 LEADER]: Becoming Leader. State: Replica: aae752b28b11458688b0565b6cf704cc, State: Running, Role: LEADER
I20260810 00:58:08.467000 26187 consensus_queue.cc:237] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aae752b28b11458688b0565b6cf704cc" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 35673 } }
I20260810 00:58:08.467768 26178 sys_catalog.cc:564] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:58:08.469937 26188 sys_catalog.cc:455] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aae752b28b11458688b0565b6cf704cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aae752b28b11458688b0565b6cf704cc" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 35673 } } }
I20260810 00:58:08.470156 26189 sys_catalog.cc:455] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader aae752b28b11458688b0565b6cf704cc. Latest consensus state: current_term: 1 leader_uuid: "aae752b28b11458688b0565b6cf704cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aae752b28b11458688b0565b6cf704cc" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 35673 } } }
I20260810 00:58:08.470381 26188 sys_catalog.cc:458] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:08.470489 26189 sys_catalog.cc:458] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:08.472222 26195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:58:08.477468 26195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:58:08.486486 26195 catalog_manager.cc:1347] Generated new cluster ID: 79893b8dd3b34e5b9b75648950a9ad2b
I20260810 00:58:08.486603 26195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:58:08.513561 26195 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:58:08.513994 26195 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:58:08.527891 26195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc: Generated new TSK 0
I20260810 00:58:08.528774 26195 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:58:08.545624 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:35673
--builtin_ntp_servers=127.20.81.148:41073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 00:58:08.883728 26209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:08.884003 26209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:08.884092 26209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:08.891204 26209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:08.891299 26209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:08.891400 26209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 00:58:08.900179 26209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:41073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:35673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:08.901397 26209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:08.903205 26209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:08.916580 26218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:08.916647 26216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:08.916671 26215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:08.917501 26209 server_base.cc:1034] running on GCE node
I20260810 00:58:08.918120 26209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:08.919114 26209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:08.920351 26209 hybrid_clock.cc:648] HybridClock initialized: now 1786323488920304 us; error 40 us; skew 500 ppm
I20260810 00:58:08.920674 26209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:58:08.923430 26209 webserver.cc:460] Webserver started at http://127.20.81.129:37873/ using document root <none> and password file <none>
I20260810 00:58:08.924136 26209 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:08.924273 26209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:08.924619 26209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:08.927556 26209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "fc176ef9b9d44e1fbf4ab78a050999f7"
format_stamp: "Formatted at 2026-08-10 00:58:08 on dist-test-slave-gp6n"
server_key: "be40b4061a33527a45c01534cf122be4"
server_key_iv: "31d0e66a075f1f765ecf04e52dcf6c4f"
server_key_version: "encryptionkey@0"
I20260810 00:58:08.928457 26209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "fc176ef9b9d44e1fbf4ab78a050999f7"
format_stamp: "Formatted at 2026-08-10 00:58:08 on dist-test-slave-gp6n"
server_key: "be40b4061a33527a45c01534cf122be4"
server_key_iv: "31d0e66a075f1f765ecf04e52dcf6c4f"
server_key_version: "encryptionkey@0"
I20260810 00:58:08.933596 26209 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260810 00:58:08.937121 26224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:08.938609 26209 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 00:58:08.938779 26209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "fc176ef9b9d44e1fbf4ab78a050999f7"
format_stamp: "Formatted at 2026-08-10 00:58:08 on dist-test-slave-gp6n"
server_key: "be40b4061a33527a45c01534cf122be4"
server_key_iv: "31d0e66a075f1f765ecf04e52dcf6c4f"
server_key_version: "encryptionkey@0"
I20260810 00:58:08.938995 26209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:08.944545 26209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:08.945523 26209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:08.945849 26209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:08.946918 26209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:08.948559 26209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:08.948685 26209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:08.948810 26209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:08.948906 26209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:08.982731 26209 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:42729
I20260810 00:58:08.982774 26350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:42729 every 8 connection(s)
I20260810 00:58:08.984052 26209 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 00:58:08.989183 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 26209
I20260810 00:58:08.989427 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 00:58:08.990079 20806 external_mini_cluster.cc:1428] Setting key 946a9e2c301978506fea3f1ee53801ce
I20260810 00:58:08.993106 26353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:08.986747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:60651 (local address 127.20.81.190:35673)
0810 00:58:08.987210 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:08.987215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:08.988621 (+ 1406us) server_negotiation.cc:407] Connection header received
0810 00:58:08.990563 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:08.990569 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:08.990646 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:08.990784 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:08.992294 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:08.992300 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:08.992304 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:58:08.992323 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:08.992377 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:58:08.992395 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:08.992526 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:08.992699 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 00:58:08.992763 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":240,"threads_started":1}
I20260810 00:58:08.993470 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:35673
--builtin_ntp_servers=127.20.81.148:41073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 00:58:08.994025 26354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:08.987467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:35673 (local address 127.20.81.129:60651)
0810 00:58:08.988443 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:08.988490 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:08.990228 (+ 1738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:08.991051 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:08.991067 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:08.991160 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:08.991898 (+ 738us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:58:08.991901 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:08.991956 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:58:08.992098 (+ 142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:08.992499 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:08.992507 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:08.992512 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 00:58:08.992727 (+ 215us) client_negotiation.cc:240] Negotiation successful
0810 00:58:08.992930 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":301,"threads_started":1}
I20260810 00:58:08.995741 26351 heartbeater.cc:347] Connected to a master server at 127.20.81.190:35673
I20260810 00:58:08.996043 26351 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:08.996627 26351 heartbeater.cc:511] Master 127.20.81.190:35673 requested a full tablet report, sending...
I20260810 00:58:08.998926 26136 ts_manager.cc:194] Registered new tserver with Master: fc176ef9b9d44e1fbf4ab78a050999f7 (127.20.81.129:42729)
W20260810 00:58:09.325695 26355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:09.325999 26355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:09.326093 26355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:09.333388 26355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:09.333498 26355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:09.333613 26355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 00:58:09.343055 26355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:41073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:35673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:09.344391 26355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:09.346267 26355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:09.359745 26360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:09.359788 26363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:09.359846 26361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:09.360608 26355 server_base.cc:1034] running on GCE node
I20260810 00:58:09.361218 26355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:09.362174 26355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:09.363404 26355 hybrid_clock.cc:648] HybridClock initialized: now 1786323489363324 us; error 73 us; skew 500 ppm
I20260810 00:58:09.363770 26355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:58:09.366134 26355 webserver.cc:460] Webserver started at http://127.20.81.130:36263/ using document root <none> and password file <none>
I20260810 00:58:09.366958 26355 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:09.367050 26355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:09.367328 26355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:09.370797 26355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "7236b74e8a894632b200398a8279bc13"
format_stamp: "Formatted at 2026-08-10 00:58:09 on dist-test-slave-gp6n"
server_key: "81026ffa61cc2dae80a08f49931710d2"
server_key_iv: "61fd8ebb478d83c0062f35c920530806"
server_key_version: "encryptionkey@0"
I20260810 00:58:09.371560 26355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "7236b74e8a894632b200398a8279bc13"
format_stamp: "Formatted at 2026-08-10 00:58:09 on dist-test-slave-gp6n"
server_key: "81026ffa61cc2dae80a08f49931710d2"
server_key_iv: "61fd8ebb478d83c0062f35c920530806"
server_key_version: "encryptionkey@0"
I20260810 00:58:09.376889 26355 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 00:58:09.380370 26370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:09.381712 26355 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 00:58:09.381901 26355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "7236b74e8a894632b200398a8279bc13"
format_stamp: "Formatted at 2026-08-10 00:58:09 on dist-test-slave-gp6n"
server_key: "81026ffa61cc2dae80a08f49931710d2"
server_key_iv: "61fd8ebb478d83c0062f35c920530806"
server_key_version: "encryptionkey@0"
I20260810 00:58:09.382128 26355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:09.388123 26355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:09.388902 26355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:09.389185 26355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:09.390148 26355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:09.391592 26355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:09.391713 26355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:09.391845 26355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:09.391940 26355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:09.426419 26355 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:46387
I20260810 00:58:09.426472 26505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:46387 every 8 connection(s)
I20260810 00:58:09.427757 26355 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 00:58:09.437184 26353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:09.430524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:35599 (local address 127.20.81.190:35673)
0810 00:58:09.430748 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:09.430755 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:09.432407 (+ 1652us) server_negotiation.cc:407] Connection header received
0810 00:58:09.434874 (+ 2467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:09.434878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:09.434957 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:09.435041 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:09.436509 (+ 1468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:09.436517 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:09.436525 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:58:09.436547 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:09.436588 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:58:09.436610 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:09.436705 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:09.436926 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 00:58:09.436974 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 00:58:09.437847 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 26355
I20260810 00:58:09.438037 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 00:58:09.438612 20806 external_mini_cluster.cc:1428] Setting key ab2845d04be60784aa8aa563b93d3af8
I20260810 00:58:09.439059 26509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:09.431124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:35673 (local address 127.20.81.130:35599)
0810 00:58:09.432195 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:09.432240 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:09.434551 (+ 2311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:09.435310 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:09.435321 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:09.435442 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:09.436158 (+ 716us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:58:09.436162 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:09.436211 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:58:09.436323 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:09.436707 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:09.436717 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:09.436725 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 00:58:09.437008 (+ 283us) client_negotiation.cc:240] Negotiation successful
0810 00:58:09.437273 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":305,"threads_started":1}
I20260810 00:58:09.440815 26507 heartbeater.cc:347] Connected to a master server at 127.20.81.190:35673
I20260810 00:58:09.441160 26507 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:09.441905 26507 heartbeater.cc:511] Master 127.20.81.190:35673 requested a full tablet report, sending...
I20260810 00:58:09.442028 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:35673
--builtin_ntp_servers=127.20.81.148:41073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 00:58:09.443586 26136 ts_manager.cc:194] Registered new tserver with Master: 7236b74e8a894632b200398a8279bc13 (127.20.81.130:46387)
W20260810 00:58:09.778204 26510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:09.778573 26510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:09.778753 26510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:09.786203 26510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:09.786309 26510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:09.786556 26510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 00:58:09.795742 26510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:41073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:35673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:09.797077 26510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:09.798925 26510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:09.812685 26515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:09.812637 26516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:09.812602 26518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:09.813584 26510 server_base.cc:1034] running on GCE node
I20260810 00:58:09.814177 26510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:09.815200 26510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:09.816439 26510 hybrid_clock.cc:648] HybridClock initialized: now 1786323489816385 us; error 67 us; skew 500 ppm
I20260810 00:58:09.816761 26510 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:58:09.819059 26510 webserver.cc:460] Webserver started at http://127.20.81.131:37215/ using document root <none> and password file <none>
I20260810 00:58:09.819749 26510 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:09.819886 26510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:09.820214 26510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:09.823140 26510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "2e25993d79c54571b9c2992042e2551d"
format_stamp: "Formatted at 2026-08-10 00:58:09 on dist-test-slave-gp6n"
server_key: "ca01e100a1c9e080a89034445b628bff"
server_key_iv: "8316280fd82e28a3b6605b438019141d"
server_key_version: "encryptionkey@0"
I20260810 00:58:09.824033 26510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "2e25993d79c54571b9c2992042e2551d"
format_stamp: "Formatted at 2026-08-10 00:58:09 on dist-test-slave-gp6n"
server_key: "ca01e100a1c9e080a89034445b628bff"
server_key_iv: "8316280fd82e28a3b6605b438019141d"
server_key_version: "encryptionkey@0"
I20260810 00:58:09.829119 26510 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 00:58:09.832618 26525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:09.833961 26510 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 00:58:09.834153 26510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "2e25993d79c54571b9c2992042e2551d"
format_stamp: "Formatted at 2026-08-10 00:58:09 on dist-test-slave-gp6n"
server_key: "ca01e100a1c9e080a89034445b628bff"
server_key_iv: "8316280fd82e28a3b6605b438019141d"
server_key_version: "encryptionkey@0"
I20260810 00:58:09.834486 26510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:09.839988 26510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:09.840754 26510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:09.841029 26510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:09.841928 26510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:09.843585 26510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:09.843715 26510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:09.843838 26510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:09.843933 26510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:09.877377 26510 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:40589
I20260810 00:58:09.877431 26660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:40589 every 8 connection(s)
I20260810 00:58:09.878798 26510 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 00:58:09.886288 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 26510
I20260810 00:58:09.886574 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 00:58:09.887146 20806 external_mini_cluster.cc:1428] Setting key e02bcb2a8be3caaa82ba1e6e7148a1d5
I20260810 00:58:09.887946 26353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:09.881570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:46355 (local address 127.20.81.190:35673)
0810 00:58:09.881761 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:09.881766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:09.883173 (+ 1407us) server_negotiation.cc:407] Connection header received
0810 00:58:09.885010 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:09.885017 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:09.885112 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:09.885236 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:09.886999 (+ 1763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:09.887005 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:09.887009 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:58:09.887028 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:09.887063 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:58:09.887077 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:09.887250 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:09.887548 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 00:58:09.887626 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 00:58:09.889081 26663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:09.881964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:35673 (local address 127.20.81.131:46355)
0810 00:58:09.883000 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:09.883039 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:09.884708 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:09.885488 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:09.885497 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:09.885595 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:09.886497 (+ 902us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:58:09.886500 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:09.886543 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:58:09.886675 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:09.887212 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:09.887224 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:09.887233 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 00:58:09.887624 (+ 391us) client_negotiation.cc:240] Negotiation successful
0810 00:58:09.887901 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":302,"threads_started":1}
I20260810 00:58:09.890684 26661 heartbeater.cc:347] Connected to a master server at 127.20.81.190:35673
I20260810 00:58:09.890981 26661 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:09.891523 26661 heartbeater.cc:511] Master 127.20.81.190:35673 requested a full tablet report, sending...
I20260810 00:58:09.892859 26135 ts_manager.cc:194] Registered new tserver with Master: 2e25993d79c54571b9c2992042e2551d (127.20.81.131:40589)
I20260810 00:58:09.903025 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:58:09.910970 26353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:09.908085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60800 (local address 127.20.81.190:35673)
0810 00:58:09.908324 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:09.908334 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:09.908767 (+ 433us) server_negotiation.cc:407] Connection header received
0810 00:58:09.908903 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:09.908922 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:09.908977 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:09.909061 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:09.910235 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:09.910243 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:09.910250 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:58:09.910270 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:09.910308 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:58:09.910324 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:09.910509 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:09.910613 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 00:58:09.910662 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 00:58:09.915217 26135 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:58:09.921974 26135 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:58:09.940775 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:09.934253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:46387 (local address 127.0.0.1:42524)
0810 00:58:09.936565 (+ 2312us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:09.936607 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:09.936730 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:09.937795 (+ 1065us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:09.937802 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:09.937894 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:09.938766 (+ 872us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:58:09.938769 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:09.938797 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:58:09.938882 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:09.940062 (+ 1180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:09.940074 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:09.940082 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 00:58:09.940239 (+ 157us) client_negotiation.cc:240] Negotiation successful
0810 00:58:09.940328 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2089,"thread_start_us":2878,"threads_started":1}
I20260810 00:58:09.941013 26678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:09.934864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60126 (local address 127.20.81.129:42729)
0810 00:58:09.935981 (+ 1117us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:09.935994 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:09.937512 (+ 1518us) server_negotiation.cc:407] Connection header received
0810 00:58:09.937756 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:09.937764 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:09.937980 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:09.938111 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:09.940099 (+ 1988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:09.940107 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:09.940118 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:58:09.940147 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:09.940206 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:58:09.940228 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:09.940326 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:09.940582 (+ 256us) server_negotiation.cc:299] Negotiation successful
0810 00:58:09.940682 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":922,"thread_start_us":811,"threads_started":1}
I20260810 00:58:09.941804 26676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:09.934286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:42729 (local address 127.0.0.1:60126)
0810 00:58:09.937408 (+ 3122us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:09.937426 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:09.937593 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:09.939152 (+ 1559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:09.939160 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:09.939186 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:09.939865 (+ 679us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:58:09.939868 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:09.939883 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:58:09.939932 (+ 49us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:09.940413 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:09.940417 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:09.940421 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 00:58:09.941247 (+ 826us) client_negotiation.cc:240] Negotiation successful
0810 00:58:09.941300 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2989,"thread_start_us":1906,"threads_started":1}
I20260810 00:58:09.942348 26677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:09.934348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42524 (local address 127.20.81.130:46387)
0810 00:58:09.937219 (+ 2871us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:09.937229 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:09.937261 (+ 32us) server_negotiation.cc:407] Connection header received
0810 00:58:09.937342 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:09.937350 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:09.937602 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:09.938317 (+ 715us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:09.939805 (+ 1488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:09.939815 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:09.939830 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:58:09.939858 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:09.939912 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:58:09.939932 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:09.941731 (+ 1799us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:09.941808 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 00:58:09.942004 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2680,"thread_start_us":2541,"threads_started":1}
I20260810 00:58:09.946029 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:09.938856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:40589 (local address 127.0.0.1:54824)
0810 00:58:09.941453 (+ 2597us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:09.941469 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:09.941549 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:09.943987 (+ 2438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:09.943992 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:09.944031 (+ 39us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:09.945118 (+ 1087us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:58:09.945121 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:09.945136 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:58:09.945177 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:09.945613 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:09.945624 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:09.945628 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 00:58:09.945710 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 00:58:09.945755 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2486,"thread_start_us":4079,"threads_started":1}
I20260810 00:58:09.946156 26279 tablet_service.cc:1467] Processing CreateTablet for tablet 41d46e940028420a945a4277a372e846 (DEFAULT_TABLE table=test-table [id=98f2b7cc2cdf455a8a0b2e0c71fbdd2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:09.946930 26679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:09.937867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54824 (local address 127.20.81.131:40589)
0810 00:58:09.943531 (+ 5664us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:09.943539 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:09.943568 (+ 29us) server_negotiation.cc:407] Connection header received
0810 00:58:09.943638 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:09.943645 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:09.943829 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:09.943995 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:09.945363 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:09.945371 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:09.945383 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:58:09.945403 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:09.945463 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:58:09.945483 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:09.946512 (+ 1029us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:09.946563 (+ 51us) server_negotiation.cc:299] Negotiation successful
0810 00:58:09.946630 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5503,"thread_start_us":5685,"threads_started":1}
I20260810 00:58:09.947211 26427 tablet_service.cc:1467] Processing CreateTablet for tablet 41d46e940028420a945a4277a372e846 (DEFAULT_TABLE table=test-table [id=98f2b7cc2cdf455a8a0b2e0c71fbdd2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:09.947760 26279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41d46e940028420a945a4277a372e846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:09.948585 26427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41d46e940028420a945a4277a372e846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:09.949637 26583 tablet_service.cc:1467] Processing CreateTablet for tablet 41d46e940028420a945a4277a372e846 (DEFAULT_TABLE table=test-table [id=98f2b7cc2cdf455a8a0b2e0c71fbdd2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:09.950984 26583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41d46e940028420a945a4277a372e846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:09.961555 26681 tablet_bootstrap.cc:492] T 41d46e940028420a945a4277a372e846 P 2e25993d79c54571b9c2992042e2551d: Bootstrap starting.
I20260810 00:58:09.965086 26681 tablet_bootstrap.cc:654] T 41d46e940028420a945a4277a372e846 P 2e25993d79c54571b9c2992042e2551d: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:09.966095 26681 log.cc:824] T 41d46e940028420a945a4277a372e846 P 2e25993d79c54571b9c2992042e2551d: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:09.969300 26681 tablet_bootstrap.cc:492] T 41d46e940028420a945a4277a372e846 P 2e25993d79c54571b9c2992042e2551d: No bootstrap required, opened a new log
I20260810 00:58:09.969563 26681 ts_tablet_manager.cc:1397] T 41d46e940028420a945a4277a372e846 P 2e25993d79c54571b9c2992042e2551d: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260810 00:58:09.971155 26683 tablet_bootstrap.cc:492] T 41d46e940028420a945a4277a372e846 P fc176ef9b9d44e1fbf4ab78a050999f7: Bootstrap starting.
I20260810 00:58:09.971302 26682 tablet_bootstrap.cc:492] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13: Bootstrap starting.
I20260810 00:58:09.975250 26682 tablet_bootstrap.cc:654] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:09.975248 26683 tablet_bootstrap.cc:654] T 41d46e940028420a945a4277a372e846 P fc176ef9b9d44e1fbf4ab78a050999f7: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:09.976514 26683 log.cc:824] T 41d46e940028420a945a4277a372e846 P fc176ef9b9d44e1fbf4ab78a050999f7: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:09.976516 26682 log.cc:824] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:09.977418 26681 raft_consensus.cc:348] T 41d46e940028420a945a4277a372e846 P 2e25993d79c54571b9c2992042e2551d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7236b74e8a894632b200398a8279bc13" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 46387 } } peers { permanent_uuid: "fc176ef9b9d44e1fbf4ab78a050999f7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 42729 } } peers { permanent_uuid: "2e25993d79c54571b9c2992042e2551d" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 40589 } }
I20260810 00:58:09.977890 26681 raft_consensus.cc:374] T 41d46e940028420a945a4277a372e846 P 2e25993d79c54571b9c2992042e2551d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:09.978050 26681 raft_consensus.cc:729] T 41d46e940028420a945a4277a372e846 P 2e25993d79c54571b9c2992042e2551d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e25993d79c54571b9c2992042e2551d, State: Initialized, Role: FOLLOWER
I20260810 00:58:09.978708 26681 consensus_queue.cc:260] T 41d46e940028420a945a4277a372e846 P 2e25993d79c54571b9c2992042e2551d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7236b74e8a894632b200398a8279bc13" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 46387 } } peers { permanent_uuid: "fc176ef9b9d44e1fbf4ab78a050999f7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 42729 } } peers { permanent_uuid: "2e25993d79c54571b9c2992042e2551d" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 40589 } }
I20260810 00:58:09.979835 26682 tablet_bootstrap.cc:492] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13: No bootstrap required, opened a new log
I20260810 00:58:09.979841 26661 heartbeater.cc:503] Master 127.20.81.190:35673 was elected leader, sending a full tablet report...
I20260810 00:58:09.980106 26682 ts_tablet_manager.cc:1397] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.004s
I20260810 00:58:09.980576 26681 ts_tablet_manager.cc:1428] T 41d46e940028420a945a4277a372e846 P 2e25993d79c54571b9c2992042e2551d: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260810 00:58:09.981177 26683 tablet_bootstrap.cc:492] T 41d46e940028420a945a4277a372e846 P fc176ef9b9d44e1fbf4ab78a050999f7: No bootstrap required, opened a new log
I20260810 00:58:09.981503 26683 ts_tablet_manager.cc:1397] T 41d46e940028420a945a4277a372e846 P fc176ef9b9d44e1fbf4ab78a050999f7: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.006s
I20260810 00:58:09.991133 26682 raft_consensus.cc:348] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7236b74e8a894632b200398a8279bc13" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 46387 } } peers { permanent_uuid: "fc176ef9b9d44e1fbf4ab78a050999f7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 42729 } } peers { permanent_uuid: "2e25993d79c54571b9c2992042e2551d" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 40589 } }
I20260810 00:58:09.991595 26682 raft_consensus.cc:374] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:09.991694 26682 raft_consensus.cc:729] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7236b74e8a894632b200398a8279bc13, State: Initialized, Role: FOLLOWER
I20260810 00:58:09.992340 26682 consensus_queue.cc:260] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7236b74e8a894632b200398a8279bc13" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 46387 } } peers { permanent_uuid: "fc176ef9b9d44e1fbf4ab78a050999f7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 42729 } } peers { permanent_uuid: "2e25993d79c54571b9c2992042e2551d" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 40589 } }
I20260810 00:58:09.993479 26507 heartbeater.cc:503] Master 127.20.81.190:35673 was elected leader, sending a full tablet report...
I20260810 00:58:09.993443 26683 raft_consensus.cc:348] T 41d46e940028420a945a4277a372e846 P fc176ef9b9d44e1fbf4ab78a050999f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7236b74e8a894632b200398a8279bc13" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 46387 } } peers { permanent_uuid: "fc176ef9b9d44e1fbf4ab78a050999f7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 42729 } } peers { permanent_uuid: "2e25993d79c54571b9c2992042e2551d" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 40589 } }
I20260810 00:58:09.993898 26683 raft_consensus.cc:374] T 41d46e940028420a945a4277a372e846 P fc176ef9b9d44e1fbf4ab78a050999f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:09.993996 26683 raft_consensus.cc:729] T 41d46e940028420a945a4277a372e846 P fc176ef9b9d44e1fbf4ab78a050999f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc176ef9b9d44e1fbf4ab78a050999f7, State: Initialized, Role: FOLLOWER
I20260810 00:58:09.994107 26682 ts_tablet_manager.cc:1428] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260810 00:58:09.994736 26683 consensus_queue.cc:260] T 41d46e940028420a945a4277a372e846 P fc176ef9b9d44e1fbf4ab78a050999f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7236b74e8a894632b200398a8279bc13" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 46387 } } peers { permanent_uuid: "fc176ef9b9d44e1fbf4ab78a050999f7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 42729 } } peers { permanent_uuid: "2e25993d79c54571b9c2992042e2551d" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 40589 } }
I20260810 00:58:09.997453 26351 heartbeater.cc:503] Master 127.20.81.190:35673 was elected leader, sending a full tablet report...
I20260810 00:58:09.997525 26683 ts_tablet_manager.cc:1428] T 41d46e940028420a945a4277a372e846 P fc176ef9b9d44e1fbf4ab78a050999f7: Time spent starting tablet: real 0.016s user 0.006s sys 0.007s
W20260810 00:58:10.130523 26662 tablet.cc:2369] T 41d46e940028420a945a4277a372e846 P 2e25993d79c54571b9c2992042e2551d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:58:10.152971 26688 raft_consensus.cc:482] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:58:10.153369 26688 raft_consensus.cc:504] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7236b74e8a894632b200398a8279bc13" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 46387 } } peers { permanent_uuid: "fc176ef9b9d44e1fbf4ab78a050999f7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 42729 } } peers { permanent_uuid: "2e25993d79c54571b9c2992042e2551d" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 40589 } }
I20260810 00:58:10.155459 26688 leader_election.cc:288] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc176ef9b9d44e1fbf4ab78a050999f7 (127.20.81.129:42729), 2e25993d79c54571b9c2992042e2551d (127.20.81.131:40589)
I20260810 00:58:10.159502 26679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:10.155966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:33099 (local address 127.20.81.131:40589)
0810 00:58:10.156204 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:10.156212 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:10.157067 (+ 855us) server_negotiation.cc:407] Connection header received
0810 00:58:10.157230 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:10.157234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:10.157284 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:10.157456 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:10.158786 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:10.158792 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:10.158797 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:58:10.158823 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:10.158862 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:58:10.158875 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:10.159010 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:10.159119 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 00:58:10.159193 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 00:58:10.159530 26678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:10.155849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:49661 (local address 127.20.81.129:42729)
0810 00:58:10.156099 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:10.156127 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:10.156999 (+ 872us) server_negotiation.cc:407] Connection header received
0810 00:58:10.157202 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:10.157207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:10.157267 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:10.157350 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:10.158832 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:10.158836 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:10.158839 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:58:10.158856 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:10.158889 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:58:10.158901 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:10.159040 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:10.159183 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 00:58:10.159246 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 00:58:10.159745 26691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:10.155769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:40589 (local address 127.20.81.130:33099)
0810 00:58:10.156901 (+ 1132us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:10.156917 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:10.157064 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:10.157494 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:10.157499 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:10.157527 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:10.158560 (+ 1033us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:58:10.158563 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:10.158577 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:58:10.158617 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:10.158986 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:10.158993 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:10.158998 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 00:58:10.159202 (+ 204us) client_negotiation.cc:240] Negotiation successful
0810 00:58:10.159254 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1026,"thread_start_us":740,"threads_started":1}
I20260810 00:58:10.159890 26690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:10.155719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:42729 (local address 127.20.81.130:49661)
0810 00:58:10.156861 (+ 1142us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:10.156880 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:10.157044 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:10.157458 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:10.157463 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:10.157491 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:10.158582 (+ 1091us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:58:10.158585 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:10.158599 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:58:10.158636 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:10.159036 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:10.159042 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:10.159048 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 00:58:10.159220 (+ 172us) client_negotiation.cc:240] Negotiation successful
0810 00:58:10.159271 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"thread_start_us":714,"threads_started":1}
I20260810 00:58:10.160907 26607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41d46e940028420a945a4277a372e846" candidate_uuid: "7236b74e8a894632b200398a8279bc13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e25993d79c54571b9c2992042e2551d" is_pre_election: true
I20260810 00:58:10.160897 26301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41d46e940028420a945a4277a372e846" candidate_uuid: "7236b74e8a894632b200398a8279bc13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc176ef9b9d44e1fbf4ab78a050999f7" is_pre_election: true
I20260810 00:58:10.161433 26607 raft_consensus.cc:2454] T 41d46e940028420a945a4277a372e846 P 2e25993d79c54571b9c2992042e2551d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7236b74e8a894632b200398a8279bc13 in term 0.
I20260810 00:58:10.161468 26301 raft_consensus.cc:2454] T 41d46e940028420a945a4277a372e846 P fc176ef9b9d44e1fbf4ab78a050999f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7236b74e8a894632b200398a8279bc13 in term 0.
I20260810 00:58:10.162144 26374 leader_election.cc:302] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7236b74e8a894632b200398a8279bc13, fc176ef9b9d44e1fbf4ab78a050999f7; no voters:
I20260810 00:58:10.162596 26688 raft_consensus.cc:2781] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:58:10.162751 26688 raft_consensus.cc:482] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:58:10.162883 26688 raft_consensus.cc:3037] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:10.165148 26688 raft_consensus.cc:504] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7236b74e8a894632b200398a8279bc13" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 46387 } } peers { permanent_uuid: "fc176ef9b9d44e1fbf4ab78a050999f7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 42729 } } peers { permanent_uuid: "2e25993d79c54571b9c2992042e2551d" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 40589 } }
I20260810 00:58:10.165756 26688 leader_election.cc:288] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13 [CANDIDATE]: Term 1 election: Requested vote from peers fc176ef9b9d44e1fbf4ab78a050999f7 (127.20.81.129:42729), 2e25993d79c54571b9c2992042e2551d (127.20.81.131:40589)
I20260810 00:58:10.166268 26301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41d46e940028420a945a4277a372e846" candidate_uuid: "7236b74e8a894632b200398a8279bc13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc176ef9b9d44e1fbf4ab78a050999f7"
I20260810 00:58:10.166290 26607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41d46e940028420a945a4277a372e846" candidate_uuid: "7236b74e8a894632b200398a8279bc13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e25993d79c54571b9c2992042e2551d"
I20260810 00:58:10.166524 26301 raft_consensus.cc:3037] T 41d46e940028420a945a4277a372e846 P fc176ef9b9d44e1fbf4ab78a050999f7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:10.166533 26607 raft_consensus.cc:3037] T 41d46e940028420a945a4277a372e846 P 2e25993d79c54571b9c2992042e2551d [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:10.169025 26301 raft_consensus.cc:2454] T 41d46e940028420a945a4277a372e846 P fc176ef9b9d44e1fbf4ab78a050999f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7236b74e8a894632b200398a8279bc13 in term 1.
I20260810 00:58:10.169430 26374 leader_election.cc:302] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7236b74e8a894632b200398a8279bc13, fc176ef9b9d44e1fbf4ab78a050999f7; no voters:
I20260810 00:58:10.169464 26607 raft_consensus.cc:2454] T 41d46e940028420a945a4277a372e846 P 2e25993d79c54571b9c2992042e2551d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7236b74e8a894632b200398a8279bc13 in term 1.
I20260810 00:58:10.169787 26688 raft_consensus.cc:2781] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:58:10.170380 26688 raft_consensus.cc:686] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13 [term 1 LEADER]: Becoming Leader. State: Replica: 7236b74e8a894632b200398a8279bc13, State: Running, Role: LEADER
I20260810 00:58:10.171011 26688 consensus_queue.cc:237] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7236b74e8a894632b200398a8279bc13" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 46387 } } peers { permanent_uuid: "fc176ef9b9d44e1fbf4ab78a050999f7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 42729 } } peers { permanent_uuid: "2e25993d79c54571b9c2992042e2551d" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 40589 } }
I20260810 00:58:10.176062 26135 catalog_manager.cc:5495] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7236b74e8a894632b200398a8279bc13 (127.20.81.130). New cstate: current_term: 1 leader_uuid: "7236b74e8a894632b200398a8279bc13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7236b74e8a894632b200398a8279bc13" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 46387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc176ef9b9d44e1fbf4ab78a050999f7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 42729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e25993d79c54571b9c2992042e2551d" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 40589 } health_report { overall_health: UNKNOWN } } }
W20260810 00:58:10.179930 26508 tablet.cc:2369] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:58:10.236557 26352 tablet.cc:2369] T 41d46e940028420a945a4277a372e846 P fc176ef9b9d44e1fbf4ab78a050999f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:58:10.238523 26677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:10.236098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42540 (local address 127.20.81.130:46387)
0810 00:58:10.236319 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:10.236326 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:10.236350 (+ 24us) server_negotiation.cc:407] Connection header received
0810 00:58:10.236582 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:10.236590 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:10.236710 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:10.236879 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:10.237863 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:10.237867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:10.237871 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:58:10.237886 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:10.237926 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:58:10.237938 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:10.238030 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:10.238119 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 00:58:10.238211 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 00:58:10.247131 26301 raft_consensus.cc:1261] T 41d46e940028420a945a4277a372e846 P fc176ef9b9d44e1fbf4ab78a050999f7 [term 1 FOLLOWER]: Refusing update from remote peer 7236b74e8a894632b200398a8279bc13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:58:10.247131 26607 raft_consensus.cc:1261] T 41d46e940028420a945a4277a372e846 P 2e25993d79c54571b9c2992042e2551d [term 1 FOLLOWER]: Refusing update from remote peer 7236b74e8a894632b200398a8279bc13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:58:10.248272 26692 consensus_queue.cc:1035] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc176ef9b9d44e1fbf4ab78a050999f7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 42729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:58:10.248720 26688 consensus_queue.cc:1035] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e25993d79c54571b9c2992042e2551d" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 40589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:58:10.260877 26696 mvcc.cc:205] Tried to move back new op lower bound from 7316781016040689664 to 7316781015745044480. Current Snapshot: MvccSnapshot[applied={T|T < 7316781016040689664}]
I20260810 00:58:10.276965 26135 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:58:10.277256 26135 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:58:10.280862 26135 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aae752b28b11458688b0565b6cf704cc: Sending DeleteTablet for 3 replicas of tablet 41d46e940028420a945a4277a372e846
I20260810 00:58:10.282212 26427 tablet_service.cc:1514] Processing DeleteTablet for tablet 41d46e940028420a945a4277a372e846 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:10 UTC) from {username='slave'} at 127.0.0.1:42524
I20260810 00:58:10.282478 26279 tablet_service.cc:1514] Processing DeleteTablet for tablet 41d46e940028420a945a4277a372e846 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:10 UTC) from {username='slave'} at 127.0.0.1:60126
I20260810 00:58:10.283221 26583 tablet_service.cc:1514] Processing DeleteTablet for tablet 41d46e940028420a945a4277a372e846 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:10 UTC) from {username='slave'} at 127.0.0.1:54824
I20260810 00:58:10.283485 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 26209
I20260810 00:58:10.296314 26708 tablet_replica.cc:330] T 41d46e940028420a945a4277a372e846 P 2e25993d79c54571b9c2992042e2551d: stopping tablet replica
I20260810 00:58:10.298297 26708 raft_consensus.cc:2229] T 41d46e940028420a945a4277a372e846 P 2e25993d79c54571b9c2992042e2551d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:58:10.299531 26707 tablet_replica.cc:330] T 41d46e940028420a945a4277a372e846 P fc176ef9b9d44e1fbf4ab78a050999f7: stopping tablet replica
I20260810 00:58:10.299856 26708 raft_consensus.cc:2258] T 41d46e940028420a945a4277a372e846 P 2e25993d79c54571b9c2992042e2551d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:10.300302 26707 raft_consensus.cc:2229] T 41d46e940028420a945a4277a372e846 P fc176ef9b9d44e1fbf4ab78a050999f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:58:10.300573 26678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:10.285199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60130 (local address 127.20.81.129:42729)
0810 00:58:10.285397 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:10.285403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:10.296668 (+ 11265us) server_negotiation.cc:407] Connection header received
0810 00:58:10.297039 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:10.297046 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:10.297169 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:10.297249 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:10.298948 (+ 1699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:10.298954 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:10.298965 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:58:10.298987 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:10.299027 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:58:10.299047 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:10.299159 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:10.300105 (+ 946us) server_negotiation.cc:299] Negotiation successful
0810 00:58:10.300178 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 00:58:10.300949 26707 raft_consensus.cc:2258] T 41d46e940028420a945a4277a372e846 P fc176ef9b9d44e1fbf4ab78a050999f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:10.301826 26706 tablet_replica.cc:330] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13: stopping tablet replica
I20260810 00:58:10.301903 26346 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:10.302358 26706 raft_consensus.cc:2229] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:58:10.303313 26706 raft_consensus.cc:2258] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:10.306638 26706 ts_tablet_manager.cc:1905] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:10.308483 26708 ts_tablet_manager.cc:1905] T 41d46e940028420a945a4277a372e846 P 2e25993d79c54571b9c2992042e2551d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:10.315065 26706 ts_tablet_manager.cc:1918] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:58:10.315382 26706 log.cc:1196] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/41d46e940028420a945a4277a372e846
I20260810 00:58:10.316143 26706 ts_tablet_manager.cc:1939] T 41d46e940028420a945a4277a372e846 P 7236b74e8a894632b200398a8279bc13: Deleting consensus metadata
I20260810 00:58:10.316130 26708 ts_tablet_manager.cc:1918] T 41d46e940028420a945a4277a372e846 P 2e25993d79c54571b9c2992042e2551d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:58:10.316280 26708 log.cc:1196] T 41d46e940028420a945a4277a372e846 P 2e25993d79c54571b9c2992042e2551d: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/41d46e940028420a945a4277a372e846
I20260810 00:58:10.316875 26708 ts_tablet_manager.cc:1939] T 41d46e940028420a945a4277a372e846 P 2e25993d79c54571b9c2992042e2551d: Deleting consensus metadata
I20260810 00:58:10.318667 26120 catalog_manager.cc:4841] TS 7236b74e8a894632b200398a8279bc13 (127.20.81.130:46387): tablet 41d46e940028420a945a4277a372e846 (table test-table [id=98f2b7cc2cdf455a8a0b2e0c71fbdd2f]) successfully deleted
I20260810 00:58:10.319092 26120 catalog_manager.cc:4841] TS 2e25993d79c54571b9c2992042e2551d (127.20.81.131:40589): tablet 41d46e940028420a945a4277a372e846 (table test-table [id=98f2b7cc2cdf455a8a0b2e0c71fbdd2f]) successfully deleted
I20260810 00:58:10.396059 26707 ts_tablet_manager.cc:1905] T 41d46e940028420a945a4277a372e846 P fc176ef9b9d44e1fbf4ab78a050999f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:10.399096 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 26209
I20260810 00:58:10.421569 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 26355
W20260810 00:58:10.421586 26121 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.81.129:42729 (error 108)
I20260810 00:58:10.423487 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:10.422983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:42729 (local address 127.0.0.1:60138)
0810 00:58:10.423157 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:10.423291 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:42729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 00:58:10.423835 26121 catalog_manager.cc:4584] TS fc176ef9b9d44e1fbf4ab78a050999f7 (127.20.81.129:42729): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 41d46e940028420a945a4277a372e846: Network error: Client connection negotiation failed: client connection to 127.20.81.129:42729: connect: Connection refused (error 111)
I20260810 00:58:10.425367 26677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:10.423033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42552 (local address 127.20.81.130:46387)
0810 00:58:10.423224 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:10.423229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:10.423247 (+ 18us) server_negotiation.cc:407] Connection header received
0810 00:58:10.423349 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:10.423353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:10.423403 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:10.423547 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:10.424565 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:10.424571 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:10.424580 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:58:10.424609 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:10.424640 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:58:10.424653 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:10.424798 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:10.424977 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 00:58:10.425045 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 00:58:10.426203 26499 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:10.477478 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:10.476978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:42729 (local address 127.0.0.1:60148)
0810 00:58:10.477210 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:10.477323 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:42729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 00:58:10.478621 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:10.478188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:42729 (local address 127.0.0.1:60156)
0810 00:58:10.478341 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:10.478510 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:42729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 00:58:10.539657 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:10.539024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:42729 (local address 127.0.0.1:60166)
0810 00:58:10.539337 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:10.539459 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:42729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260810 00:58:10.539733 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 26355
I20260810 00:58:10.540977 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:10.540583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:42729 (local address 127.0.0.1:60180)
0810 00:58:10.540758 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:10.540849 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:42729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 00:58:10.561962 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 26510
I20260810 00:58:10.565096 26679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:10.562844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54826 (local address 127.20.81.131:40589)
0810 00:58:10.563034 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:10.563040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:10.563124 (+ 84us) server_negotiation.cc:407] Connection header received
0810 00:58:10.563338 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:10.563343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:10.563444 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:10.563559 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:10.564439 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:10.564444 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:10.564447 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:58:10.564462 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:10.564493 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:58:10.564504 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:10.564640 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:10.564722 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 00:58:10.564791 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 00:58:10.565793 26655 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:10.622165 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:10.621695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:42729 (local address 127.0.0.1:60190)
0810 00:58:10.621923 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:10.622013 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:42729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260810 00:58:10.623308 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:10.622868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:42729 (local address 127.0.0.1:60194)
0810 00:58:10.623078 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:10.623150 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:42729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260810 00:58:10.654026 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 26510
I20260810 00:58:10.673594 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 26103
I20260810 00:58:10.674750 26172 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:10.781025 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 26103
2026-08-10T00:58:10Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2857 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 00:58:10.827754 20806 mini_oidc.cc:152] Starting JWKS server
I20260810 00:58:10.830618 20806 webserver.cc:460] Webserver started at https://127.0.0.1:44561/ using document root <none> and password file <none>
I20260810 00:58:10.830897 20806 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 00:58:10.831544 20806 webserver.cc:460] Webserver started at http://127.0.0.1:42557/ using document root <none> and password file <none>
2026-08-10T00:58:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:58:10Z Disabled control of system clock
I20260810 00:58:10.864889 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:44161
--webserver_interface=127.20.81.190
--webserver_port=0
--builtin_ntp_servers=127.20.81.148:44029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:44161
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44561/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323445691443-20806-0/testchainca.cert with env {}
W20260810 00:58:11.197142 26722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:11.197501 26722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:11.197671 26722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:11.204654 26722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:58:11.204800 26722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:11.204897 26722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:11.205006 26722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:58:11.205078 26722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 00:58:11.205168 26722 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 00:58:11.205250 26722 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44561/jwks.json
I20260810 00:58:11.214843 26722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:44161
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:44161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44561/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323445691443-20806-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:11.216111 26722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:11.217440 26722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:11.229725 26728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:11.229770 26731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:11.229807 26729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:11.230443 26722 server_base.cc:1034] running on GCE node
I20260810 00:58:11.231266 26722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:11.232650 26722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:11.233878 26722 hybrid_clock.cc:648] HybridClock initialized: now 1786323491233834 us; error 44 us; skew 500 ppm
I20260810 00:58:11.234243 26722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:58:11.236675 26722 webserver.cc:460] Webserver started at http://127.20.81.190:43195/ using document root <none> and password file <none>
I20260810 00:58:11.237367 26722 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:11.237489 26722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:11.237828 26722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:11.240772 26722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "90e055367e974edbbde4f73e4b4b72cd"
format_stamp: "Formatted at 2026-08-10 00:58:11 on dist-test-slave-gp6n"
server_key: "cfa13418527e18abb637457f8a6d49b8"
server_key_iv: "cf05fefc2fadc07fde2b77b04e4c180b"
server_key_version: "encryptionkey@0"
I20260810 00:58:11.241567 26722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "90e055367e974edbbde4f73e4b4b72cd"
format_stamp: "Formatted at 2026-08-10 00:58:11 on dist-test-slave-gp6n"
server_key: "cfa13418527e18abb637457f8a6d49b8"
server_key_iv: "cf05fefc2fadc07fde2b77b04e4c180b"
server_key_version: "encryptionkey@0"
I20260810 00:58:11.246569 26722 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260810 00:58:11.249943 26738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:11.251410 26722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:58:11.251602 26722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "90e055367e974edbbde4f73e4b4b72cd"
format_stamp: "Formatted at 2026-08-10 00:58:11 on dist-test-slave-gp6n"
server_key: "cfa13418527e18abb637457f8a6d49b8"
server_key_iv: "cf05fefc2fadc07fde2b77b04e4c180b"
server_key_version: "encryptionkey@0"
I20260810 00:58:11.251833 26722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:11.275497 26722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:11.276369 26722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:11.276695 26722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:11.296037 26722 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:44161
I20260810 00:58:11.296072 26793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:44161 every 8 connection(s)
I20260810 00:58:11.297431 26722 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:58:11.301251 26794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:11.306511 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 26722
I20260810 00:58:11.306715 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 00:58:11.307547 20806 external_mini_cluster.cc:1428] Setting key e58b1e32785432819c1d6f55a0476392
I20260810 00:58:11.313199 26794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd: Bootstrap starting.
I20260810 00:58:11.316910 26794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:11.318032 26794 log.cc:824] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:11.321125 26794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd: No bootstrap required, opened a new log
I20260810 00:58:11.321925 26797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:11.309634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33796 (local address 127.20.81.190:44161)
0810 00:58:11.310770 (+ 1136us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:11.310785 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:11.310829 (+ 44us) server_negotiation.cc:407] Connection header received
0810 00:58:11.312539 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:11.312563 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:11.313014 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:11.313398 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:11.314035 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:11.315504 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:11.317365 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:11.317996 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:11.318139 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:11.319580 (+ 1441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:11.319588 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:11.319601 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:58:11.320103 (+ 502us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:11.320175 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:58:11.320194 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:11.320286 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:11.320537 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 00:58:11.320858 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":444,"threads_started":1}
I20260810 00:58:11.325701 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:58:11.329891 26794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e055367e974edbbde4f73e4b4b72cd" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44161 } }
I20260810 00:58:11.330212 26794 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:11.330282 26794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e055367e974edbbde4f73e4b4b72cd, State: Initialized, Role: FOLLOWER
I20260810 00:58:11.330811 26794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e055367e974edbbde4f73e4b4b72cd" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44161 } }
I20260810 00:58:11.331013 26794 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:58:11.331140 26794 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:58:11.331280 26794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:11.333731 26794 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e055367e974edbbde4f73e4b4b72cd" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44161 } }
I20260810 00:58:11.334194 26794 leader_election.cc:302] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90e055367e974edbbde4f73e4b4b72cd; no voters:
I20260810 00:58:11.334786 26794 leader_election.cc:288] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:58:11.334960 26799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:58:11.335686 26799 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd [term 1 LEADER]: Becoming Leader. State: Replica: 90e055367e974edbbde4f73e4b4b72cd, State: Running, Role: LEADER
I20260810 00:58:11.336259 26799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e055367e974edbbde4f73e4b4b72cd" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44161 } }
I20260810 00:58:11.336889 26794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:58:11.339773 26801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90e055367e974edbbde4f73e4b4b72cd. Latest consensus state: current_term: 1 leader_uuid: "90e055367e974edbbde4f73e4b4b72cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e055367e974edbbde4f73e4b4b72cd" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44161 } } }
I20260810 00:58:11.339494 26800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90e055367e974edbbde4f73e4b4b72cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e055367e974edbbde4f73e4b4b72cd" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44161 } } }
I20260810 00:58:11.340592 26801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:11.340716 26800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:11.342190 26804 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:58:11.347196 26804 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:58:11.357189 26804 catalog_manager.cc:1347] Generated new cluster ID: 7c532917db0c40408a83253392f7ef49
I20260810 00:58:11.357331 26804 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:58:11.377360 26804 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:58:11.379314 26804 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:58:11.396068 26804 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90e055367e974edbbde4f73e4b4b72cd: Generated new TSK 0
I20260810 00:58:11.397089 26804 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 00:58:11.444926 26825 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 00:58:11.445191 26825 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 00:58:11.446308 26797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:11.422342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55746 (local address 127.20.81.190:44161)
0810 00:58:11.422627 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:11.422634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:11.423098 (+ 464us) server_negotiation.cc:407] Connection header received
0810 00:58:11.423493 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:11.423499 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:11.423618 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:11.423781 (+ 163us) server_negotiation.cc:226] Negotiated authn=JWT
0810 00:58:11.424454 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:11.425252 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:11.427435 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:11.427936 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:11.428038 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:11.428690 (+ 652us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 00:58:11.445732 (+ 17042us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44561/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 00:58:11.446063 (+ 331us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55746: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44561/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":92}
W20260810 00:58:11.446542 26797 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55746: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44561/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 00:58:11.448354 26824 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:11.422234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44161 (local address 127.0.0.1:55746)
0810 00:58:11.422957 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:11.422984 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:11.423180 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:11.423845 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:11.423854 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:11.423871 (+ 17us) client_negotiation.cc:189] Negotiated authn=JWT
0810 00:58:11.424266 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:11.424279 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:11.425412 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:11.425418 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:11.427264 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:11.427280 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:11.428125 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:11.428133 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:11.428316 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:11.428392 (+ 76us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 00:58:11.446048 (+ 17656us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44561/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 00:58:11.446467 (+ 419us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.81.190:44161: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44561/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":355,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.20.81.190:44161: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44561/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 00:58:11.450799 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 26722
I20260810 00:58:11.452020 26789 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:11.548818 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 26722
2026-08-10T00:58:11Z chronyd exiting
I20260810 00:58:11.602093 20806 test_util.cc:175] -----------------------------------------------
I20260810 00:58:11.602250 20806 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323445691443-20806-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (779 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 00:58:11.606138 20806 mini_oidc.cc:152] Starting JWKS server
I20260810 00:58:11.608076 20806 webserver.cc:460] Webserver started at https://127.0.0.1:46861/ using document root <none> and password file <none>
I20260810 00:58:11.608340 20806 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 00:58:11.609020 20806 webserver.cc:460] Webserver started at http://127.0.0.1:35719/ using document root <none> and password file <none>
2026-08-10T00:58:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:58:11Z Disabled control of system clock
I20260810 00:58:11.642241 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:42863
--webserver_interface=127.20.81.190
--webserver_port=0
--builtin_ntp_servers=127.20.81.148:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:42863
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46861/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323445691443-20806-0/testchainca.cert with env {}
W20260810 00:58:11.971726 26832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:11.972100 26832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:11.972260 26832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:11.979302 26832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:58:11.979452 26832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:11.979548 26832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:11.979660 26832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:58:11.979748 26832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 00:58:11.979838 26832 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 00:58:11.979918 26832 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46861/jwks.json
I20260810 00:58:11.989468 26832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:42863
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:42863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46861/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323445691443-20806-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:11.990782 26832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:11.992101 26832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:12.004041 26838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:12.004081 26837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:12.004200 26841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:12.005287 26832 server_base.cc:1034] running on GCE node
I20260810 00:58:12.006086 26832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:12.007714 26832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:12.008956 26832 hybrid_clock.cc:648] HybridClock initialized: now 1786323492008915 us; error 65 us; skew 500 ppm
I20260810 00:58:12.009326 26832 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:58:12.011631 26832 webserver.cc:460] Webserver started at http://127.20.81.190:37543/ using document root <none> and password file <none>
I20260810 00:58:12.012326 26832 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:12.012467 26832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:12.012796 26832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:12.015669 26832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "709ca69616814b73955c49a5840e2da1"
format_stamp: "Formatted at 2026-08-10 00:58:12 on dist-test-slave-gp6n"
server_key: "e247632146ca86ed0aa5946471e8593a"
server_key_iv: "44eb323dfe04b18a93b676d3d2ee428f"
server_key_version: "encryptionkey@0"
I20260810 00:58:12.016458 26832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "709ca69616814b73955c49a5840e2da1"
format_stamp: "Formatted at 2026-08-10 00:58:12 on dist-test-slave-gp6n"
server_key: "e247632146ca86ed0aa5946471e8593a"
server_key_iv: "44eb323dfe04b18a93b676d3d2ee428f"
server_key_version: "encryptionkey@0"
I20260810 00:58:12.021370 26832 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 00:58:12.024673 26849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:12.026001 26832 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 00:58:12.026192 26832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "709ca69616814b73955c49a5840e2da1"
format_stamp: "Formatted at 2026-08-10 00:58:12 on dist-test-slave-gp6n"
server_key: "e247632146ca86ed0aa5946471e8593a"
server_key_iv: "44eb323dfe04b18a93b676d3d2ee428f"
server_key_version: "encryptionkey@0"
I20260810 00:58:12.026403 26832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:12.041239 26832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:12.042068 26832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:12.042376 26832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:12.062042 26832 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:42863
I20260810 00:58:12.062062 26905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:42863 every 8 connection(s)
I20260810 00:58:12.063478 26832 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:58:12.067339 26906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:12.073482 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 26832
I20260810 00:58:12.073717 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 00:58:12.074577 20806 external_mini_cluster.cc:1428] Setting key c86d490b6ce0acc7208fbe4e5bc27310
I20260810 00:58:12.078768 26906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1: Bootstrap starting.
I20260810 00:58:12.081871 26906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:12.083025 26906 log.cc:824] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:12.086231 26906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1: No bootstrap required, opened a new log
I20260810 00:58:12.088630 26909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:12.076570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34358 (local address 127.20.81.190:42863)
0810 00:58:12.077524 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:12.077543 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:12.077585 (+ 42us) server_negotiation.cc:407] Connection header received
0810 00:58:12.079357 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:12.079380 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:12.079822 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:12.080233 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:12.080799 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:12.082118 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:12.083586 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:12.084419 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:12.084612 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:12.085730 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:12.085741 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:12.085760 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:58:12.086514 (+ 754us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:12.086615 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:58:12.086641 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:12.086785 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:12.086988 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 00:58:12.087314 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":317,"threads_started":1}
I20260810 00:58:12.092674 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:58:12.095432 26906 raft_consensus.cc:348] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709ca69616814b73955c49a5840e2da1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 42863 } }
I20260810 00:58:12.095808 26906 raft_consensus.cc:374] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:12.095933 26906 raft_consensus.cc:729] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 709ca69616814b73955c49a5840e2da1, State: Initialized, Role: FOLLOWER
I20260810 00:58:12.096546 26906 consensus_queue.cc:260] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709ca69616814b73955c49a5840e2da1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 42863 } }
I20260810 00:58:12.096753 26906 raft_consensus.cc:388] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:58:12.096851 26906 raft_consensus.cc:482] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:58:12.096984 26906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:12.099361 26906 raft_consensus.cc:504] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709ca69616814b73955c49a5840e2da1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 42863 } }
I20260810 00:58:12.099821 26906 leader_election.cc:302] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 709ca69616814b73955c49a5840e2da1; no voters:
I20260810 00:58:12.100337 26906 leader_election.cc:288] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:58:12.100534 26911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:58:12.101399 26911 raft_consensus.cc:686] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1 [term 1 LEADER]: Becoming Leader. State: Replica: 709ca69616814b73955c49a5840e2da1, State: Running, Role: LEADER
I20260810 00:58:12.101881 26911 consensus_queue.cc:237] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709ca69616814b73955c49a5840e2da1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 42863 } }
I20260810 00:58:12.102634 26906 sys_catalog.cc:564] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:58:12.104846 26912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "709ca69616814b73955c49a5840e2da1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709ca69616814b73955c49a5840e2da1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 42863 } } }
I20260810 00:58:12.105266 26912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:12.106098 26913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 709ca69616814b73955c49a5840e2da1. Latest consensus state: current_term: 1 leader_uuid: "709ca69616814b73955c49a5840e2da1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709ca69616814b73955c49a5840e2da1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 42863 } } }
I20260810 00:58:12.106683 26913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:12.107817 26920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:58:12.112735 26920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:58:12.120725 26920 catalog_manager.cc:1347] Generated new cluster ID: d8caaca9bd344113ba1fecb78139ce97
I20260810 00:58:12.120841 26920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:58:12.163851 26920 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:58:12.165227 26920 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:58:12.181527 26920 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 709ca69616814b73955c49a5840e2da1: Generated new TSK 0
I20260810 00:58:12.182574 26920 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:58:12.219229 26909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:12.196382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34362 (local address 127.20.81.190:42863)
0810 00:58:12.196586 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:12.196593 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:12.197111 (+ 518us) server_negotiation.cc:407] Connection header received
0810 00:58:12.197327 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:12.197333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:12.197434 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:12.197610 (+ 176us) server_negotiation.cc:226] Negotiated authn=JWT
0810 00:58:12.198290 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:12.199459 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:12.201006 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:12.201691 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:12.201830 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:12.202286 (+ 456us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 00:58:12.218494 (+ 16208us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46861/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 00:58:12.218911 (+ 417us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34362: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46861/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":63}
W20260810 00:58:12.219503 26909 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34362: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46861/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
E20260810 00:58:12.219873 26938 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 00:58:12.220013 26938 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260810 00:58:12.220494 26937 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:12.196238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:42863 (local address 127.0.0.1:34362)
0810 00:58:12.197001 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:12.197020 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:12.197146 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:12.197659 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:12.197666 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:12.197686 (+ 20us) client_negotiation.cc:189] Negotiated authn=JWT
0810 00:58:12.198083 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:12.198097 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:12.199628 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:12.199633 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:12.200851 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:12.200863 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:12.201858 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:12.201863 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:12.201992 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:12.202026 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 00:58:12.218848 (+ 16822us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46861/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 00:58:12.219242 (+ 394us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.81.190:42863: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46861/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":445,"threads_started":1}
I20260810 00:58:12.222657 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 26832
I20260810 00:58:12.224385 26901 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:12.322764 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 26832
2026-08-10T00:58:12Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (779 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 00:58:12.385751 20806 mini_oidc.cc:152] Starting JWKS server
I20260810 00:58:12.387821 20806 webserver.cc:460] Webserver started at https://127.0.0.1:39499/ using document root <none> and password file <none>
I20260810 00:58:12.388103 20806 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 00:58:12.388828 20806 webserver.cc:460] Webserver started at http://127.0.0.1:33427/ using document root <none> and password file <none>
2026-08-10T00:58:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:58:12Z Disabled control of system clock
I20260810 00:58:12.423661 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:46821
--webserver_interface=127.20.81.190
--webserver_port=0
--builtin_ntp_servers=127.20.81.148:38973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:46821
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39499/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 00:58:12.754089 26947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:12.754489 26947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:12.754655 26947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:12.762058 26947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:58:12.762216 26947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:12.762310 26947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:12.762454 26947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:58:12.762528 26947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 00:58:12.762619 26947 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 00:58:12.762699 26947 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39499/jwks.json
I20260810 00:58:12.772123 26947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:38973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:46821
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:46821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39499/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:12.773533 26947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:12.775388 26947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:12.787436 26953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:12.787359 26955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:12.787382 26957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:12.788290 26947 server_base.cc:1034] running on GCE node
I20260810 00:58:12.789127 26947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:12.790560 26947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:12.791780 26947 hybrid_clock.cc:648] HybridClock initialized: now 1786323492791731 us; error 46 us; skew 500 ppm
I20260810 00:58:12.792130 26947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:58:12.794473 26947 webserver.cc:460] Webserver started at http://127.20.81.190:35587/ using document root <none> and password file <none>
I20260810 00:58:12.795181 26947 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:12.795317 26947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:12.795647 26947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:12.798537 26947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "2f1a8fba20c34b7d88007e1e27b2da2f"
format_stamp: "Formatted at 2026-08-10 00:58:12 on dist-test-slave-gp6n"
server_key: "fa05b98e4f9c43c5a0e3b13013ff3242"
server_key_iv: "ab7730e035eba40c7400e85e6b820b02"
server_key_version: "encryptionkey@0"
I20260810 00:58:12.799352 26947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "2f1a8fba20c34b7d88007e1e27b2da2f"
format_stamp: "Formatted at 2026-08-10 00:58:12 on dist-test-slave-gp6n"
server_key: "fa05b98e4f9c43c5a0e3b13013ff3242"
server_key_iv: "ab7730e035eba40c7400e85e6b820b02"
server_key_version: "encryptionkey@0"
I20260810 00:58:12.804315 26947 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 00:58:12.807626 26963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:12.808838 26947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 00:58:12.809026 26947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "2f1a8fba20c34b7d88007e1e27b2da2f"
format_stamp: "Formatted at 2026-08-10 00:58:12 on dist-test-slave-gp6n"
server_key: "fa05b98e4f9c43c5a0e3b13013ff3242"
server_key_iv: "ab7730e035eba40c7400e85e6b820b02"
server_key_version: "encryptionkey@0"
I20260810 00:58:12.809240 26947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:12.838722 26947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:12.839591 26947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:12.839918 26947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:12.859068 26947 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:46821
I20260810 00:58:12.859117 27024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:46821 every 8 connection(s)
I20260810 00:58:12.860401 26947 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:58:12.864050 27025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:12.864799 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 26947
I20260810 00:58:12.865002 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 00:58:12.865679 20806 external_mini_cluster.cc:1428] Setting key d02f93a465b669ef8ac99b1a39d51868
I20260810 00:58:12.878593 27025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f: Bootstrap starting.
I20260810 00:58:12.880613 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:12.867690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36666 (local address 127.20.81.190:46821)
0810 00:58:12.868802 (+ 1112us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:12.868830 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:12.868883 (+ 53us) server_negotiation.cc:407] Connection header received
0810 00:58:12.870671 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:12.870700 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:12.871305 (+ 605us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:12.871734 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:12.872262 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:12.874124 (+ 1862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:12.875484 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:12.876577 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:12.876749 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:12.877828 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:12.877838 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:12.877855 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:58:12.878572 (+ 717us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:12.878674 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:58:12.878709 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:12.878821 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:12.879027 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 00:58:12.879363 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":354,"threads_started":1}
I20260810 00:58:12.882244 27025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:12.883414 27025 log.cc:824] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:12.884498 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:58:12.886615 27025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f: No bootstrap required, opened a new log
I20260810 00:58:12.894074 27025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1a8fba20c34b7d88007e1e27b2da2f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 46821 } }
I20260810 00:58:12.894382 27025 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:12.894518 27025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f1a8fba20c34b7d88007e1e27b2da2f, State: Initialized, Role: FOLLOWER
I20260810 00:58:12.895056 27025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1a8fba20c34b7d88007e1e27b2da2f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 46821 } }
I20260810 00:58:12.895256 27025 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:58:12.895350 27025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:58:12.895468 27025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:12.897706 27025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1a8fba20c34b7d88007e1e27b2da2f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 46821 } }
I20260810 00:58:12.898149 27025 leader_election.cc:302] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f1a8fba20c34b7d88007e1e27b2da2f; no voters:
I20260810 00:58:12.898763 27025 leader_election.cc:288] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:58:12.898905 27030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:58:12.899712 27030 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f [term 1 LEADER]: Becoming Leader. State: Replica: 2f1a8fba20c34b7d88007e1e27b2da2f, State: Running, Role: LEADER
I20260810 00:58:12.900240 27030 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1a8fba20c34b7d88007e1e27b2da2f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 46821 } }
I20260810 00:58:12.900879 27025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:58:12.903057 27032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f1a8fba20c34b7d88007e1e27b2da2f. Latest consensus state: current_term: 1 leader_uuid: "2f1a8fba20c34b7d88007e1e27b2da2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1a8fba20c34b7d88007e1e27b2da2f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 46821 } } }
I20260810 00:58:12.903496 27032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:12.904079 27031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f1a8fba20c34b7d88007e1e27b2da2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1a8fba20c34b7d88007e1e27b2da2f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 46821 } } }
I20260810 00:58:12.904469 27031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:12.905197 27040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:58:12.910261 27040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:58:12.919369 27040 catalog_manager.cc:1347] Generated new cluster ID: b40378e095cf41eb8e37c6131215ddcd
I20260810 00:58:12.919523 27040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:58:12.951623 27040 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:58:12.953070 27040 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:58:12.960321 27040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f1a8fba20c34b7d88007e1e27b2da2f: Generated new TSK 0
I20260810 00:58:12.961151 27040 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:58:12.992268 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:12.983837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36682 (local address 127.20.81.190:46821)
0810 00:58:12.984028 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:12.984035 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:12.984431 (+ 396us) server_negotiation.cc:407] Connection header received
0810 00:58:12.984659 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:12.984665 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:12.984756 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:12.984925 (+ 169us) server_negotiation.cc:226] Negotiated authn=JWT
0810 00:58:12.985371 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:12.986542 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:12.987862 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:12.988573 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:12.988698 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:12.989239 (+ 541us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 00:58:12.991639 (+ 2400us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39499/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 00:58:12.991952 (+ 313us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36682: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39499/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":62}
W20260810 00:58:12.992401 27028 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36682: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39499/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 00:58:12.994020 27059 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:12.983695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:46821 (local address 127.0.0.1:36682)
0810 00:58:12.984332 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:12.984349 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:12.984467 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:12.984940 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:12.984943 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:12.984952 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0810 00:58:12.985232 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:12.985239 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:12.986679 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:12.986683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:12.987723 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:12.987734 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:12.988749 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:12.988754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:12.988935 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:12.988968 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 00:58:12.991841 (+ 2873us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39499/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 00:58:12.993608 (+ 1767us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.81.190:46821: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39499/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":1481,"thread_start_us":372,"threads_started":1}
I20260810 00:58:12.995325 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 26947
I20260810 00:58:12.996567 27018 generic_service.cc:196] Checking for leaks (request via RPC)
E20260810 00:58:13.007232 27060 webserver.cc:533] Webserver: sslize failed:
E20260810 00:58:13.007372 27060 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 00:58:13.096612 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 26947
2026-08-10T00:58:13Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (769 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T00:58:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:58:13Z Disabled control of system clock
I20260810 00:58:13.189229 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:43983
--webserver_interface=127.20.81.190
--webserver_port=0
--builtin_ntp_servers=127.20.81.148:37675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:43983
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323445691443-20806-0/testchainca.cert with env {}
W20260810 00:58:13.525465 27066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:13.525750 27066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:13.525835 27066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:13.533046 27066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:58:13.533146 27066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:13.533183 27066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:13.533245 27066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:58:13.533288 27066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 00:58:13.533350 27066 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 00:58:13.533388 27066 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 00:58:13.543192 27066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:37675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:43983
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:43983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323445691443-20806-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:13.544413 27066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:13.545732 27066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:13.557869 27071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:13.557809 27072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:13.557834 27074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:13.559032 27066 server_base.cc:1034] running on GCE node
I20260810 00:58:13.559957 27066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:13.561378 27066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:13.562598 27066 hybrid_clock.cc:648] HybridClock initialized: now 1786323493562547 us; error 50 us; skew 500 ppm
I20260810 00:58:13.562943 27066 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:58:13.569952 27066 webserver.cc:460] Webserver started at http://127.20.81.190:39225/ using document root <none> and password file <none>
I20260810 00:58:13.570761 27066 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:13.570899 27066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:13.571226 27066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:13.574110 27066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "9bc5a746d83645a49f160762c09e18b8"
format_stamp: "Formatted at 2026-08-10 00:58:13 on dist-test-slave-gp6n"
server_key: "d03baa413e2389077af12c6f2e6af0b4"
server_key_iv: "4fe5172f36e02aa6281ca3ca011b3a11"
server_key_version: "encryptionkey@0"
I20260810 00:58:13.575001 27066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "9bc5a746d83645a49f160762c09e18b8"
format_stamp: "Formatted at 2026-08-10 00:58:13 on dist-test-slave-gp6n"
server_key: "d03baa413e2389077af12c6f2e6af0b4"
server_key_iv: "4fe5172f36e02aa6281ca3ca011b3a11"
server_key_version: "encryptionkey@0"
I20260810 00:58:13.580075 27066 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260810 00:58:13.583400 27082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:13.584656 27066 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 00:58:13.584839 27066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "9bc5a746d83645a49f160762c09e18b8"
format_stamp: "Formatted at 2026-08-10 00:58:13 on dist-test-slave-gp6n"
server_key: "d03baa413e2389077af12c6f2e6af0b4"
server_key_iv: "4fe5172f36e02aa6281ca3ca011b3a11"
server_key_version: "encryptionkey@0"
I20260810 00:58:13.585062 27066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:13.602037 27066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:13.602837 27066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:13.603133 27066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:13.621837 27066 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:43983
I20260810 00:58:13.621886 27140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:43983 every 8 connection(s)
I20260810 00:58:13.623234 27066 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:58:13.627002 27141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:13.629967 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 27066
I20260810 00:58:13.630192 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 00:58:13.631045 20806 external_mini_cluster.cc:1428] Setting key fa11806b1409a32d50db06450440da9e
I20260810 00:58:13.639757 27141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8: Bootstrap starting.
I20260810 00:58:13.643283 27141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:13.644455 27141 log.cc:824] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:13.645040 27145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:13.633119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45916 (local address 127.20.81.190:43983)
0810 00:58:13.634088 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:13.634098 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:13.634140 (+ 42us) server_negotiation.cc:407] Connection header received
0810 00:58:13.635807 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:13.635830 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:13.636290 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:13.636699 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:13.637294 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:13.638664 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:13.640478 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:13.641194 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:13.641416 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:13.642688 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:13.642695 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:13.642708 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:58:13.643227 (+ 519us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:13.643318 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:58:13.643339 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:13.643467 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:13.643678 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 00:58:13.643986 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":346,"threads_started":1}
I20260810 00:58:13.647656 27141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8: No bootstrap required, opened a new log
I20260810 00:58:13.648909 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:58:13.655689 27141 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bc5a746d83645a49f160762c09e18b8" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43983 } }
I20260810 00:58:13.656081 27141 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:13.656198 27141 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc5a746d83645a49f160762c09e18b8, State: Initialized, Role: FOLLOWER
I20260810 00:58:13.656805 27141 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bc5a746d83645a49f160762c09e18b8" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43983 } }
I20260810 00:58:13.657061 27141 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:58:13.657208 27141 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:58:13.657378 27141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:13.659869 27141 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bc5a746d83645a49f160762c09e18b8" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43983 } }
I20260810 00:58:13.660410 27141 leader_election.cc:302] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bc5a746d83645a49f160762c09e18b8; no voters:
I20260810 00:58:13.661096 27141 leader_election.cc:288] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:58:13.661253 27148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:58:13.662158 27148 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8 [term 1 LEADER]: Becoming Leader. State: Replica: 9bc5a746d83645a49f160762c09e18b8, State: Running, Role: LEADER
I20260810 00:58:13.662705 27148 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bc5a746d83645a49f160762c09e18b8" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43983 } }
I20260810 00:58:13.663393 27141 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:58:13.665704 27149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9bc5a746d83645a49f160762c09e18b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bc5a746d83645a49f160762c09e18b8" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43983 } } }
I20260810 00:58:13.665726 27150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bc5a746d83645a49f160762c09e18b8. Latest consensus state: current_term: 1 leader_uuid: "9bc5a746d83645a49f160762c09e18b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bc5a746d83645a49f160762c09e18b8" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43983 } } }
I20260810 00:58:13.666116 27150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:13.666116 27149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:13.669734 27155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:58:13.675318 27155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:58:13.683858 27155 catalog_manager.cc:1347] Generated new cluster ID: f93a3d6b55a044b89c523cae4c8af2c1
I20260810 00:58:13.683983 27155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:58:13.704164 27155 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:58:13.705729 27155 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:58:13.713723 27155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9bc5a746d83645a49f160762c09e18b8: Generated new TSK 0
I20260810 00:58:13.714588 27155 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:58:14.159876 27145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:13.727705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45932 (local address 127.20.81.190:43983)
0810 00:58:13.727979 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:13.727989 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:13.728486 (+ 497us) server_negotiation.cc:407] Connection header received
0810 00:58:13.728770 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:13.728781 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:13.728854 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:13.728992 (+ 138us) server_negotiation.cc:226] Negotiated authn=JWT
0810 00:58:13.729552 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:13.730338 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:13.732390 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:13.732816 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:13.732921 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:13.733524 (+ 603us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 00:58:14.159308 (+425784us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 00:58:14.159652 (+ 344us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45932: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":85}
W20260810 00:58:14.160034 27145 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45932: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260810 00:58:14.160301 27174 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:13.727681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:43983 (local address 127.0.0.1:45932)
0810 00:58:13.728369 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:13.728389 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:13.728557 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:13.729042 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:13.729047 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:13.729060 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0810 00:58:13.729383 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:13.729398 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:13.730533 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:13.730539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:13.732233 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:13.732252 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:13.733000 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:13.733007 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:13.733216 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:13.733265 (+ 49us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 00:58:14.159599 (+426334us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 00:58:14.159855 (+ 256us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.81.190:43983: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":383,"threads_started":1}
I20260810 00:58:14.162184 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 27066
I20260810 00:58:14.163290 27136 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:14.272773 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 27066
2026-08-10T00:58:14Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1158 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (421 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (424 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:58:15 dist-test-slave-gp6n krb5kdc[27190](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:58:15 dist-test-slave-gp6n krb5kdc[27190](info): set up 2 sockets
Aug 10 00:58:15 dist-test-slave-gp6n krb5kdc[27190](info): commencing operation
krb5kdc: starting...
W20260810 00:58:16.287559 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 00:58:16 dist-test-slave-gp6n krb5kdc[27190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:58:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:58:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 00:58:16.523867 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:40427
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:41265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:40427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:16.855548 27207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:16.855861 27207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:16.855953 27207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:16.863165 27207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:58:16.863273 27207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:16.863322 27207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:16.863387 27207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:58:16.863425 27207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:58:16.873461 27207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:41265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:40427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:40427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:16.874828 27207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:16.876124 27207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:16.887660 27215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:16.887795 27212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:16.887823 27213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:16.888787 27207 server_base.cc:1034] running on GCE node
I20260810 00:58:16.889612 27207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:16.891023 27207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:16.892258 27207 hybrid_clock.cc:648] HybridClock initialized: now 1786323496892219 us; error 43 us; skew 500 ppm
I20260810 00:58:16.892621 27207 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:16 dist-test-slave-gp6n krb5kdc[27190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323496, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:16.895793 27207 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 00:58:16.897275 27207 webserver.cc:460] Webserver started at http://127.20.81.190:33795/ using document root <none> and password file <none>
I20260810 00:58:16.897975 27207 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:16.898144 27207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:16.898586 27207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:16.901522 27207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "63c6bc6e29d04f34ad243ff693e08948"
format_stamp: "Formatted at 2026-08-10 00:58:16 on dist-test-slave-gp6n"
server_key: "52700db764ff0b8a70b3842d4aeb1bc2"
server_key_iv: "143d783eebbdf4ac8f9e1f3fa2f7a999"
server_key_version: "encryptionkey@0"
I20260810 00:58:16.902318 27207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "63c6bc6e29d04f34ad243ff693e08948"
format_stamp: "Formatted at 2026-08-10 00:58:16 on dist-test-slave-gp6n"
server_key: "52700db764ff0b8a70b3842d4aeb1bc2"
server_key_iv: "143d783eebbdf4ac8f9e1f3fa2f7a999"
server_key_version: "encryptionkey@0"
I20260810 00:58:16.907455 27207 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 00:58:16.910969 27222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:16.912320 27207 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 00:58:16.912550 27207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "63c6bc6e29d04f34ad243ff693e08948"
format_stamp: "Formatted at 2026-08-10 00:58:16 on dist-test-slave-gp6n"
server_key: "52700db764ff0b8a70b3842d4aeb1bc2"
server_key_iv: "143d783eebbdf4ac8f9e1f3fa2f7a999"
server_key_version: "encryptionkey@0"
I20260810 00:58:16.912765 27207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:16.937981 27207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:16.942145 27207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:16.942524 27207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:16.961568 27207 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:40427
I20260810 00:58:16.961604 27281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:40427 every 8 connection(s)
I20260810 00:58:16.963043 27207 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:58:16.966959 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 27207
I20260810 00:58:16.967214 27282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:16.967250 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 00:58:16.968132 20806 external_mini_cluster.cc:1428] Setting key 785a279d4ed521a05a99ae0760c131e8
I20260810 00:58:16.981657 27282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948: Bootstrap starting.
Aug 10 00:58:16 dist-test-slave-gp6n krb5kdc[27190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:58:16.985354 27282 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:16.986500 27282 log.cc:824] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:16.989467 27282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948: No bootstrap required, opened a new log
I20260810 00:58:16.992415 27286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:16.970380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40618 (local address 127.20.81.190:40427)
0810 00:58:16.971255 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:16.971266 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:16.971359 (+ 93us) server_negotiation.cc:407] Connection header received
0810 00:58:16.973689 (+ 2330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:16.973715 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:16.974139 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:16.974568 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:16.976328 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:16.977673 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:16.979511 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:16.980198 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:16.980329 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:16.985031 (+ 4702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:16.985039 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:16.985053 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:16.985097 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:16.988340 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:16.989049 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:16.989055 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:16.989061 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:16.989225 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:16.989630 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:16.989639 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:16.989641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:16.990149 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:16.990501 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:16.990984 (+ 483us) server_negotiation.cc:299] Negotiation successful
0810 00:58:16.991398 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":400,"threads_started":1}
I20260810 00:58:16.998064 27282 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c6bc6e29d04f34ad243ff693e08948" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40427 } }
I20260810 00:58:16.998456 27282 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:16.998575 27282 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63c6bc6e29d04f34ad243ff693e08948, State: Initialized, Role: FOLLOWER
I20260810 00:58:16.999188 27282 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c6bc6e29d04f34ad243ff693e08948" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40427 } }
I20260810 00:58:16.999447 27282 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:58:16.999605 27282 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:58:16.999799 27282 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:17.002005 27282 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c6bc6e29d04f34ad243ff693e08948" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40427 } }
I20260810 00:58:17.002581 27282 leader_election.cc:302] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63c6bc6e29d04f34ad243ff693e08948; no voters:
I20260810 00:58:17.003216 27282 leader_election.cc:288] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:58:17.003469 27289 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:58:17.004748 27289 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948 [term 1 LEADER]: Becoming Leader. State: Replica: 63c6bc6e29d04f34ad243ff693e08948, State: Running, Role: LEADER
I20260810 00:58:17.005333 27289 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c6bc6e29d04f34ad243ff693e08948" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40427 } }
I20260810 00:58:17.005543 27282 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:58:17.008733 27290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63c6bc6e29d04f34ad243ff693e08948" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c6bc6e29d04f34ad243ff693e08948" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40427 } } }
I20260810 00:58:17.009146 27290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:17.009697 27292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63c6bc6e29d04f34ad243ff693e08948. Latest consensus state: current_term: 1 leader_uuid: "63c6bc6e29d04f34ad243ff693e08948" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c6bc6e29d04f34ad243ff693e08948" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40427 } } }
I20260810 00:58:17.010048 27292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:17.011526 27297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:58:17.017613 27297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:58:17.027339 27297 catalog_manager.cc:1347] Generated new cluster ID: e60440e57cf242aeb868c77e6e3a0b7a
I20260810 00:58:17.027479 27297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:58:17.046927 27297 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:58:17.048240 27297 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:58:17.054535 27297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63c6bc6e29d04f34ad243ff693e08948: Generated new TSK 0
I20260810 00:58:17.055400 27297 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.81.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
I20260810 00:58:17.122249 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:40427
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:41265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:17.488390 27312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:17.488677 27312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:17.488850 27312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:17.496326 27312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:17.496429 27312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:17.496630 27312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 00:58:17.506496 27312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:41265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:40427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:17.507886 27312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:17.509222 27312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:17.523034 27317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:17.522971 27318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:17.523068 27322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:17.523772 27312 server_base.cc:1034] running on GCE node
I20260810 00:58:17.524353 27312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:17.525252 27312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:17.526496 27312 hybrid_clock.cc:648] HybridClock initialized: now 1786323497526426 us; error 69 us; skew 500 ppm
I20260810 00:58:17.526796 27312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:17 dist-test-slave-gp6n krb5kdc[27190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323497, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:17.529951 27312 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 00:58:17.531488 27312 webserver.cc:460] Webserver started at http://127.20.81.129:36315/ using document root <none> and password file <none>
I20260810 00:58:17.532203 27312 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:17.532351 27312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:17.532693 27312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:17.536638 27312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "d1135bd208064a8980828bd71147d68e"
format_stamp: "Formatted at 2026-08-10 00:58:17 on dist-test-slave-gp6n"
server_key: "fed29aa46a8e8473b6b285f6a1558719"
server_key_iv: "c7409e16ac36bd7ac56c5443380a4a00"
server_key_version: "encryptionkey@0"
I20260810 00:58:17.537554 27312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "d1135bd208064a8980828bd71147d68e"
format_stamp: "Formatted at 2026-08-10 00:58:17 on dist-test-slave-gp6n"
server_key: "fed29aa46a8e8473b6b285f6a1558719"
server_key_iv: "c7409e16ac36bd7ac56c5443380a4a00"
server_key_version: "encryptionkey@0"
I20260810 00:58:17.542611 27312 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260810 00:58:17.545948 27329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:17.547226 27312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:58:17.547429 27312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "d1135bd208064a8980828bd71147d68e"
format_stamp: "Formatted at 2026-08-10 00:58:17 on dist-test-slave-gp6n"
server_key: "fed29aa46a8e8473b6b285f6a1558719"
server_key_iv: "c7409e16ac36bd7ac56c5443380a4a00"
server_key_version: "encryptionkey@0"
I20260810 00:58:17.547641 27312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:17.562553 27312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:17.566365 27312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:17.566720 27312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:17.567713 27312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:17.569130 27312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:17.569245 27312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:17.569415 27312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:17.569495 27312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:17.602965 27312 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:42023
I20260810 00:58:17.603015 27452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:42023 every 8 connection(s)
I20260810 00:58:17.604318 27312 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 00:58:17.610680 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 27312
I20260810 00:58:17.610906 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 00:58:17.611642 20806 external_mini_cluster.cc:1428] Setting key d4f8b08e40a4ae599c98afdc8b7fad33
Aug 10 00:58:17 dist-test-slave-gp6n krb5kdc[27190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323497, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:58:17.625357 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:17.606975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:44223 (local address 127.20.81.190:40427)
0810 00:58:17.607487 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:17.607492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:17.608378 (+ 886us) server_negotiation.cc:407] Connection header received
0810 00:58:17.610232 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:17.610237 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:17.610338 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:17.610514 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:17.612269 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:17.613304 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:17.614941 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:17.615337 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:17.615472 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:17.619621 (+ 4149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:17.619626 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:17.619629 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:17.619661 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:17.622296 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:17.623075 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:17.623082 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:17.623091 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:17.623208 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:17.623648 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:17.623652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:17.623658 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:17.623923 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:17.624080 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:17.624770 (+ 690us) server_negotiation.cc:299] Negotiation successful
0810 00:58:17.625001 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":264,"threads_started":1}
I20260810 00:58:17.626346 27456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:17.607477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:40427 (local address 127.20.81.129:44223)
0810 00:58:17.608182 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:17.608222 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:17.609933 (+ 1711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:17.610747 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:17.610755 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:17.611464 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:17.612099 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:17.612117 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:17.613498 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:17.613502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:17.614794 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:17.614806 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:17.615579 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:17.615588 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:17.615850 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:17.616588 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:17.616621 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:17.619413 (+ 2792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:17.622469 (+ 3056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:17.622478 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:17.622494 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:17.622947 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:17.623328 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:17.623333 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:17.623337 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:17.623525 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:17.624053 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:17.624061 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:17.624402 (+ 341us) client_negotiation.cc:769] Sending connection context
0810 00:58:17.624656 (+ 254us) client_negotiation.cc:240] Negotiation successful
0810 00:58:17.625171 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":310,"threads_started":1}
I20260810 00:58:17.627990 27453 heartbeater.cc:347] Connected to a master server at 127.20.81.190:40427
I20260810 00:58:17.628324 27453 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:17.629205 27453 heartbeater.cc:511] Master 127.20.81.190:40427 requested a full tablet report, sending...
I20260810 00:58:17.631872 27240 ts_manager.cc:194] Registered new tserver with Master: d1135bd208064a8980828bd71147d68e (127.20.81.129:42023)
I20260810 00:58:17.634148 27240 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:44223
WARNING: no policy specified for kudu/127.20.81.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
I20260810 00:58:17.667567 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:40427
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:41265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:18.001811 27459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:18.002077 27459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:18.002171 27459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:18.009665 27459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:18.009776 27459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:18.009891 27459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 00:58:18.019693 27459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:41265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:40427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:18.020926 27459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:18.022197 27459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:18.035328 27465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:18.035374 27469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:18.035673 27467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:18.036238 27459 server_base.cc:1034] running on GCE node
I20260810 00:58:18.036799 27459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:18.037703 27459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:18.038897 27459 hybrid_clock.cc:648] HybridClock initialized: now 1786323498038858 us; error 29 us; skew 500 ppm
I20260810 00:58:18.039220 27459 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:18 dist-test-slave-gp6n krb5kdc[27190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323498, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:18.042516 27459 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 00:58:18.043938 27459 webserver.cc:460] Webserver started at http://127.20.81.130:41579/ using document root <none> and password file <none>
I20260810 00:58:18.044621 27459 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:18.044755 27459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:18.045090 27459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:18.048018 27459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "49f839af487d4d928f1e8b504ccb8bbb"
format_stamp: "Formatted at 2026-08-10 00:58:18 on dist-test-slave-gp6n"
server_key: "0c9f3778bdd33a1368ec8fe3fc61642f"
server_key_iv: "aef9b4ac3cfa016ff30088f062a17459"
server_key_version: "encryptionkey@0"
I20260810 00:58:18.048868 27459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "49f839af487d4d928f1e8b504ccb8bbb"
format_stamp: "Formatted at 2026-08-10 00:58:18 on dist-test-slave-gp6n"
server_key: "0c9f3778bdd33a1368ec8fe3fc61642f"
server_key_iv: "aef9b4ac3cfa016ff30088f062a17459"
server_key_version: "encryptionkey@0"
I20260810 00:58:18.054270 27459 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 00:58:18.057833 27476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:18.059478 27459 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:58:18.059652 27459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "49f839af487d4d928f1e8b504ccb8bbb"
format_stamp: "Formatted at 2026-08-10 00:58:18 on dist-test-slave-gp6n"
server_key: "0c9f3778bdd33a1368ec8fe3fc61642f"
server_key_iv: "aef9b4ac3cfa016ff30088f062a17459"
server_key_version: "encryptionkey@0"
I20260810 00:58:18.059875 27459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:18.076799 27459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:18.080785 27459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:18.081069 27459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:18.082015 27459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:18.083385 27459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:18.083467 27459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:18.083623 27459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:18.083675 27459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:18.121737 27459 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:46209
I20260810 00:58:18.121778 27600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:46209 every 8 connection(s)
I20260810 00:58:18.123167 27459 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 00:58:18.130740 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 27459
I20260810 00:58:18.130957 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 00:58:18.131428 20806 external_mini_cluster.cc:1428] Setting key 26b51d5297f9103942c6a5c9d64b4e05
Aug 10 00:58:18 dist-test-slave-gp6n krb5kdc[27190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323498, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:58:18.145923 27606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:18.125785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:43893 (local address 127.20.81.190:40427)
0810 00:58:18.126245 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:18.126250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:18.127356 (+ 1106us) server_negotiation.cc:407] Connection header received
0810 00:58:18.129141 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:18.129146 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:18.129213 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:18.129328 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:18.130999 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:18.131803 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:18.133432 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:18.133849 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:18.133944 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:18.138525 (+ 4581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:18.138535 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:18.138543 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:18.138590 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:18.142232 (+ 3642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:18.143296 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:18.143301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:18.143305 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:18.143478 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:18.143919 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:18.143927 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:18.143933 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:18.144331 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:18.144487 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:18.145165 (+ 678us) server_negotiation.cc:299] Negotiation successful
0810 00:58:18.145428 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":263,"threads_started":1}
I20260810 00:58:18.146613 27607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:18.126366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:40427 (local address 127.20.81.130:43893)
0810 00:58:18.127134 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:18.127210 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:18.128888 (+ 1678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:18.129542 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:18.129549 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:18.130046 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:18.130823 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:18.130853 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:18.131970 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:18.131974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:18.133258 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:18.133274 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:18.134085 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:18.134094 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:18.134446 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:18.135212 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:18.135246 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:18.138256 (+ 3010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:18.142396 (+ 4140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:18.142441 (+ 45us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:18.142461 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:18.143132 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:18.143603 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:18.143606 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:18.143609 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:18.143791 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:18.144456 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:18.144462 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:18.144785 (+ 323us) client_negotiation.cc:769] Sending connection context
0810 00:58:18.145010 (+ 225us) client_negotiation.cc:240] Negotiation successful
0810 00:58:18.145390 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":281,"threads_started":1}
I20260810 00:58:18.148233 27603 heartbeater.cc:347] Connected to a master server at 127.20.81.190:40427
I20260810 00:58:18.148566 27603 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:18.149369 27603 heartbeater.cc:511] Master 127.20.81.190:40427 requested a full tablet report, sending...
I20260810 00:58:18.150951 27240 ts_manager.cc:194] Registered new tserver with Master: 49f839af487d4d928f1e8b504ccb8bbb (127.20.81.130:46209)
I20260810 00:58:18.151978 27240 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:43893
WARNING: no policy specified for kudu/127.20.81.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
I20260810 00:58:18.187615 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:40427
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:41265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:18.532156 27610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:18.532437 27610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:18.532662 27610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:18.541541 27610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:18.541654 27610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:18.541842 27610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 00:58:18.551977 27610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:41265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:40427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:18.553283 27610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:18.554752 27610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:18.568321 27617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:18.568197 27616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:18.568166 27619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:18.569222 27610 server_base.cc:1034] running on GCE node
I20260810 00:58:18.569936 27610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:18.571187 27610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:18.572456 27610 hybrid_clock.cc:648] HybridClock initialized: now 1786323498572349 us; error 71 us; skew 500 ppm
I20260810 00:58:18.572810 27610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:18 dist-test-slave-gp6n krb5kdc[27190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323498, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:18.580148 27610 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 00:58:18.581732 27610 webserver.cc:460] Webserver started at http://127.20.81.131:41931/ using document root <none> and password file <none>
I20260810 00:58:18.582530 27610 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:18.582679 27610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:18.583012 27610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:18.586015 27610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "09b5aaf0a2da4412aa490a0219a8f073"
format_stamp: "Formatted at 2026-08-10 00:58:18 on dist-test-slave-gp6n"
server_key: "b4c0112b05555fe6c045b759361c7b21"
server_key_iv: "efdc979952c5f3a2c69b3ad97c5f1b3e"
server_key_version: "encryptionkey@0"
I20260810 00:58:18.586845 27610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "09b5aaf0a2da4412aa490a0219a8f073"
format_stamp: "Formatted at 2026-08-10 00:58:18 on dist-test-slave-gp6n"
server_key: "b4c0112b05555fe6c045b759361c7b21"
server_key_iv: "efdc979952c5f3a2c69b3ad97c5f1b3e"
server_key_version: "encryptionkey@0"
I20260810 00:58:18.592027 27610 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 00:58:18.595520 27629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:18.596836 27610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 00:58:18.597018 27610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "09b5aaf0a2da4412aa490a0219a8f073"
format_stamp: "Formatted at 2026-08-10 00:58:18 on dist-test-slave-gp6n"
server_key: "b4c0112b05555fe6c045b759361c7b21"
server_key_iv: "efdc979952c5f3a2c69b3ad97c5f1b3e"
server_key_version: "encryptionkey@0"
I20260810 00:58:18.597225 27610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:18.620805 27610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:18.625344 27610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:18.625667 27610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:18.626729 27610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:18.628091 27610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:18.628196 27610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:18.628343 27610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:18.628424 27610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:18.638055 27453 heartbeater.cc:503] Master 127.20.81.190:40427 was elected leader, sending a full tablet report...
I20260810 00:58:18.663126 27610 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:43845
I20260810 00:58:18.663290 27752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:43845 every 8 connection(s)
I20260810 00:58:18.664503 27610 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 00:58:18.666229 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 27610
I20260810 00:58:18.666469 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 00:58:18.666927 20806 external_mini_cluster.cc:1428] Setting key 9eea3b012f7f75ccea6f9d731c36510b
Aug 10 00:58:18 dist-test-slave-gp6n krb5kdc[27190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323498, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:58:18.685612 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:18.667396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:36983 (local address 127.20.81.190:40427)
0810 00:58:18.667884 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:18.667890 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:18.669136 (+ 1246us) server_negotiation.cc:407] Connection header received
0810 00:58:18.671267 (+ 2131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:18.671271 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:18.671359 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:18.671462 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:18.673134 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:18.673915 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:18.675497 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:18.675913 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:18.676011 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:18.679315 (+ 3304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:18.679319 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:18.679322 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:18.679355 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:18.682528 (+ 3173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:18.683334 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:18.683339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:18.683342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:18.683468 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:18.683933 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:18.683937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:18.683939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:18.684240 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:18.684357 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:18.685128 (+ 771us) server_negotiation.cc:299] Negotiation successful
0810 00:58:18.685303 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":260,"threads_started":1}
I20260810 00:58:18.686748 27757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:18.667983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:40427 (local address 127.20.81.131:36983)
0810 00:58:18.668964 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:18.669008 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:18.671006 (+ 1998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:18.671857 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:18.671865 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:18.672397 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:18.672972 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:18.672989 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:18.674043 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:18.674047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:18.675361 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:18.675373 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:18.676094 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:18.676098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:18.676294 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:18.676974 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:18.676996 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:18.679129 (+ 2133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:18.682690 (+ 3561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:18.682697 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:18.682710 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:18.683212 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:18.683605 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:18.683609 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:18.683612 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:18.683814 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:18.684403 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:18.684410 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:18.684765 (+ 355us) client_negotiation.cc:769] Sending connection context
0810 00:58:18.685003 (+ 238us) client_negotiation.cc:240] Negotiation successful
0810 00:58:18.685407 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":415,"threads_started":1}
I20260810 00:58:18.688223 27753 heartbeater.cc:347] Connected to a master server at 127.20.81.190:40427
I20260810 00:58:18.688558 27753 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:18.689334 27753 heartbeater.cc:511] Master 127.20.81.190:40427 requested a full tablet report, sending...
I20260810 00:58:18.690846 27240 ts_manager.cc:194] Registered new tserver with Master: 09b5aaf0a2da4412aa490a0219a8f073 (127.20.81.131:43845)
I20260810 00:58:18.691685 27240 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:36983
I20260810 00:58:18.695336 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/test-admin.keytab.
Aug 10 00:58:18 dist-test-slave-gp6n krb5kdc[27190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:18.728112 20806 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 00:58:18.733544 20806 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/krb5cc' of type FILE
W20260810 00:58:18.733628 27765 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:18.730940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41008 (local address 127.20.81.129:42023)
0810 00:58:18.731617 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:18.731624 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:18.731651 (+ 27us) server_negotiation.cc:407] Connection header received
0810 00:58:18.731713 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:18.731720 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:18.731928 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:18.732061 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:18.733145 (+ 1084us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41008: BlockingRecv error: recv got EOF from 127.0.0.1:41008 (error 108)
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":340,"threads_started":1}
Aug 10 00:58:18 dist-test-slave-gp6n krb5kdc[27190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:18.735276 20806 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 00:58:18 dist-test-slave-gp6n krb5kdc[27190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:18.736796 20806 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 00:58:18 dist-test-slave-gp6n krb5kdc[27190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.129@KRBTEST.COM
I20260810 00:58:18.752442 27765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:18.738521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41020 (local address 127.20.81.129:42023)
0810 00:58:18.738721 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:18.738727 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:18.738887 (+ 160us) server_negotiation.cc:407] Connection header received
0810 00:58:18.739120 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:18.739132 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:18.739194 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:18.739301 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:18.740328 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:18.741289 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:18.742556 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:18.743123 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:18.743232 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:18.746893 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:18.746905 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:18.746919 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:18.746953 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:18.749674 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:18.750380 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:18.750387 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:18.750389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:18.750574 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:18.751057 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:18.751060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:18.751062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:18.751406 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:18.751655 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:18.751919 (+ 264us) server_negotiation.cc:299] Negotiation successful
0810 00:58:18.752186 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260810 00:58:18.758327 27765 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:18.756438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41032 (local address 127.20.81.129:42023)
0810 00:58:18.756652 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:18.756657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:18.756891 (+ 234us) server_negotiation.cc:407] Connection header received
0810 00:58:18.757080 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:18.757086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:18.757162 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:18.757263 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:18.758086 (+ 823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41032: BlockingRecv error: recv got EOF from 127.0.0.1:41032 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260810 00:58:18.758538 20806 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/krb5cc' of type FILE
Aug 10 00:58:18 dist-test-slave-gp6n krb5kdc[27190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:18.760104 20806 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 00:58:18 dist-test-slave-gp6n krb5kdc[27190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:18.761365 20806 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 00:58:18 dist-test-slave-gp6n krb5kdc[27190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.129@KRBTEST.COM
I20260810 00:58:18.775406 27765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:18.762600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41042 (local address 127.20.81.129:42023)
0810 00:58:18.762789 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:18.762798 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:18.763096 (+ 298us) server_negotiation.cc:407] Connection header received
0810 00:58:18.763210 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:18.763214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:18.763266 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:18.763342 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:18.764310 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:18.765102 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:18.766224 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:18.766739 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:18.766833 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:18.770423 (+ 3590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:18.770431 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:18.770437 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:18.770512 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:18.773088 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:18.773652 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:18.773656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:18.773659 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:18.773776 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:18.774131 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:18.774134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:18.774136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:18.774386 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:18.774675 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:18.774918 (+ 243us) server_negotiation.cc:299] Negotiation successful
0810 00:58:18.775098 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260810 00:58:18.779803 27765 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:18.777971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41050 (local address 127.20.81.129:42023)
0810 00:58:18.778186 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:18.778192 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:18.778470 (+ 278us) server_negotiation.cc:407] Connection header received
0810 00:58:18.778683 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:18.778688 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:18.778770 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:18.778885 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:18.779576 (+ 691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41050: BlockingRecv error: recv got EOF from 127.0.0.1:41050 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260810 00:58:18.780023 20806 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323445691443-20806-0/krb5kdc/krb5cc' of type FILE
Aug 10 00:58:18 dist-test-slave-gp6n krb5kdc[27190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:18.781556 20806 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 00:58:18 dist-test-slave-gp6n krb5kdc[27190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:18.782938 20806 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 00:58:18 dist-test-slave-gp6n krb5kdc[27190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.129@KRBTEST.COM
I20260810 00:58:18.798295 27765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:18.784387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41052 (local address 127.20.81.129:42023)
0810 00:58:18.784619 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:18.784625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:18.784885 (+ 260us) server_negotiation.cc:407] Connection header received
0810 00:58:18.784979 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:18.784983 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:18.785040 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:18.785127 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:18.786122 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:18.786960 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:18.788215 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:18.788663 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:18.788786 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:18.792768 (+ 3982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:18.792774 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:18.792777 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:18.792812 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:18.795815 (+ 3003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:18.796465 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:18.796469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:18.796472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:18.796608 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:18.796938 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:18.796941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:18.796944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:18.797215 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:18.797460 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:18.797718 (+ 258us) server_negotiation.cc:299] Negotiation successful
0810 00:58:18.797951 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 00:58:18.800232 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 27312
I20260810 00:58:18.813706 27765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:18.801073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41058 (local address 127.20.81.129:42023)
0810 00:58:18.801269 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:18.801277 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:18.801684 (+ 407us) server_negotiation.cc:407] Connection header received
0810 00:58:18.801907 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:18.801912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:18.801968 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:18.802073 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:18.803170 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:18.803957 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:18.805276 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:18.805844 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:18.805995 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:18.808293 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:18.808308 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:18.808312 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:18.808342 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:18.810854 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:18.811633 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:18.811640 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:18.811643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:18.811783 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:18.812278 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:18.812283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:18.812285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:18.812542 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:18.812824 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:18.813147 (+ 323us) server_negotiation.cc:299] Negotiation successful
0810 00:58:18.813365 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 00:58:18.814278 27448 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:18.906373 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 27312
I20260810 00:58:18.926877 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 27459
Aug 10 00:58:18 dist-test-slave-gp6n krb5kdc[27190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.130@KRBTEST.COM
I20260810 00:58:18.942144 27780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:18.927887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35434 (local address 127.20.81.130:46209)
0810 00:58:18.928373 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:18.928381 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:18.928406 (+ 25us) server_negotiation.cc:407] Connection header received
0810 00:58:18.928465 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:18.928471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:18.928684 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:18.928809 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:18.929781 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:18.930827 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:18.932037 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:18.932546 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:18.932645 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:18.936424 (+ 3779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:18.936435 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:18.936448 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:18.936479 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:18.939357 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:18.939959 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:18.939965 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:18.939968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:18.940102 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:18.940452 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:18.940457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:18.940459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:18.940849 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:18.941122 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:18.941514 (+ 392us) server_negotiation.cc:299] Negotiation successful
0810 00:58:18.941798 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":248,"threads_started":1}
I20260810 00:58:18.943718 27596 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:19.033833 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 27459
I20260810 00:58:19.053015 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 27610
Aug 10 00:58:19 dist-test-slave-gp6n krb5kdc[27190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.131@KRBTEST.COM
I20260810 00:58:19.068428 27782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:19.054065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60956 (local address 127.20.81.131:43845)
0810 00:58:19.054737 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:19.054744 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:19.054774 (+ 30us) server_negotiation.cc:407] Connection header received
0810 00:58:19.054880 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:19.054887 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:19.055102 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:19.055235 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:19.056237 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:19.057247 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:19.058479 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:19.059057 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:19.059145 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:19.063043 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:19.063054 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:19.063069 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:19.063105 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:19.065967 (+ 2862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:19.066698 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:19.066705 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:19.066708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:19.066828 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:19.067132 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:19.067135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:19.067137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:19.067450 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:19.067691 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:19.067867 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 00:58:19.068115 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":363,"threads_started":1}
I20260810 00:58:19.069689 27748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:19.151968 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 27610
I20260810 00:58:19.172151 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 27207
I20260810 00:58:19.173520 27275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:19.279042 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 27207
2026-08-10T00:58:19Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4165 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:58:19 dist-test-slave-gp6n krb5kdc[27789](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:58:19 dist-test-slave-gp6n krb5kdc[27789](info): set up 2 sockets
Aug 10 00:58:19 dist-test-slave-gp6n krb5kdc[27789](info): commencing operation
krb5kdc: starting...
W20260810 00:58:20.461074 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.003s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 00:58:20 dist-test-slave-gp6n krb5kdc[27789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:58:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:58:20Z Disabled control of system clock
WARNING: no policy specified for oryx/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.190.keytab.
Entry for principal oryx/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.190.keytab.
I20260810 00:58:20.706210 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:33681
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.190.keytab
--principal=oryx/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:40295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:33681
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:21.037853 27803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:21.038131 27803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:21.038303 27803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:21.045470 27803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:58:21.045598 27803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:21.045683 27803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:21.045786 27803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:58:21.045876 27803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:58:21.056406 27803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:40295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:33681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.190.keytab
--principal=oryx/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:33681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:21.057705 27803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:21.059105 27803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:21.070734 27808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:21.070864 27809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:21.071313 27811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:21.071849 27803 server_base.cc:1034] running on GCE node
I20260810 00:58:21.072623 27803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:21.074030 27803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:21.075273 27803 hybrid_clock.cc:648] HybridClock initialized: now 1786323501075216 us; error 46 us; skew 500 ppm
I20260810 00:58:21.075695 27803 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:21 dist-test-slave-gp6n krb5kdc[27789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323501, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:21.079473 27803 init.cc:377] Logged in from keytab as oryx/127.20.81.190@KRBTEST.COM (short username oryx)
I20260810 00:58:21.080979 27803 webserver.cc:460] Webserver started at http://127.20.81.190:40253/ using document root <none> and password file <none>
I20260810 00:58:21.081658 27803 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:21.081804 27803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:21.082108 27803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:21.085089 27803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "31ff0eea6672410291c03a7df56d1cf2"
format_stamp: "Formatted at 2026-08-10 00:58:21 on dist-test-slave-gp6n"
server_key: "1cd154b8e0e5824f8fa60ed7636065c4"
server_key_iv: "55cce3db4dc6b73847f4f9a8c146165a"
server_key_version: "encryptionkey@0"
I20260810 00:58:21.085860 27803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "31ff0eea6672410291c03a7df56d1cf2"
format_stamp: "Formatted at 2026-08-10 00:58:21 on dist-test-slave-gp6n"
server_key: "1cd154b8e0e5824f8fa60ed7636065c4"
server_key_iv: "55cce3db4dc6b73847f4f9a8c146165a"
server_key_version: "encryptionkey@0"
I20260810 00:58:21.091181 27803 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 00:58:21.094619 27821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:21.095908 27803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 00:58:21.096098 27803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "31ff0eea6672410291c03a7df56d1cf2"
format_stamp: "Formatted at 2026-08-10 00:58:21 on dist-test-slave-gp6n"
server_key: "1cd154b8e0e5824f8fa60ed7636065c4"
server_key_iv: "55cce3db4dc6b73847f4f9a8c146165a"
server_key_version: "encryptionkey@0"
I20260810 00:58:21.096295 27803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:21.111239 27803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:21.115577 27803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:21.115868 27803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:21.135344 27803 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:33681
I20260810 00:58:21.135430 27873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:33681 every 8 connection(s)
I20260810 00:58:21.136754 27803 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:58:21.138008 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 27803
I20260810 00:58:21.138283 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 00:58:21.139185 20806 external_mini_cluster.cc:1428] Setting key 36fb7e92cacfa865a58c24fd494a4fee
I20260810 00:58:21.141772 27874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:21.152933 27874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2: Bootstrap starting.
Aug 10 00:58:21 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM
I20260810 00:58:21.156836 27874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:21.157985 27874 log.cc:824] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:21.161341 27874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2: No bootstrap required, opened a new log
I20260810 00:58:21.162185 27878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:21.141391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52152 (local address 127.20.81.190:33681)
0810 00:58:21.142249 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:21.142261 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:21.142301 (+ 40us) server_negotiation.cc:407] Connection header received
0810 00:58:21.143969 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:21.143990 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:21.144378 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:21.144779 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:21.146339 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:21.147736 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:21.149016 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:21.149717 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:21.149965 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:21.154349 (+ 4384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:21.154357 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:21.154371 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:21.154467 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:21.157635 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:21.158561 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:21.158575 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:21.158586 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:21.158863 (+ 277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:21.159367 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:21.159372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:21.159375 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:21.159936 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:21.160156 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:21.160631 (+ 475us) server_negotiation.cc:299] Negotiation successful
0810 00:58:21.161028 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":411,"threads_started":1}
I20260810 00:58:21.168984 27874 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } }
I20260810 00:58:21.169377 27874 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:21.169483 27874 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31ff0eea6672410291c03a7df56d1cf2, State: Initialized, Role: FOLLOWER
I20260810 00:58:21.170010 27874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } }
I20260810 00:58:21.170256 27874 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:58:21.170395 27874 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:58:21.170630 27874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:21.172712 27874 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } }
I20260810 00:58:21.173202 27874 leader_election.cc:302] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31ff0eea6672410291c03a7df56d1cf2; no voters:
I20260810 00:58:21.173834 27874 leader_election.cc:288] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:58:21.174009 27880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:58:21.175446 27880 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [term 1 LEADER]: Becoming Leader. State: Replica: 31ff0eea6672410291c03a7df56d1cf2, State: Running, Role: LEADER
I20260810 00:58:21.176059 27880 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } }
I20260810 00:58:21.176244 27874 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:58:21.179420 27881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31ff0eea6672410291c03a7df56d1cf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } }
I20260810 00:58:21.179797 27881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:21.180289 27882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31ff0eea6672410291c03a7df56d1cf2. Latest consensus state: current_term: 1 leader_uuid: "31ff0eea6672410291c03a7df56d1cf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } }
I20260810 00:58:21.180615 27882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:21.183727 27888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:58:21.189488 27888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:58:21.198874 27888 catalog_manager.cc:1347] Generated new cluster ID: 8c10a06edd3847a3bd451ed9039cadf6
I20260810 00:58:21.198994 27888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:58:21.227267 27888 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:58:21.228762 27888 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:58:21.237421 27888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2: Generated new TSK 0
I20260810 00:58:21.238301 27888 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.20.81.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.81.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.81.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.129.keytab.
Entry for principal oryx/127.20.81.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.129.keytab.
I20260810 00:58:21.311025 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:33681
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.129.keytab
--principal=oryx/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:40295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:21.648800 27902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:21.649078 27902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:21.649238 27902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:21.656409 27902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:21.656510 27902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:21.656713 27902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 00:58:21.666251 27902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:40295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.129.keytab
--principal=oryx/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:33681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:21.667539 27902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:21.668864 27902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:21.682509 27908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:21.682561 27909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:21.682765 27912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:21.683418 27902 server_base.cc:1034] running on GCE node
I20260810 00:58:21.684047 27902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:21.685115 27902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:21.686344 27902 hybrid_clock.cc:648] HybridClock initialized: now 1786323501686324 us; error 66 us; skew 500 ppm
I20260810 00:58:21.686721 27902 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:21 dist-test-slave-gp6n krb5kdc[27789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323501, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:21.690016 27902 init.cc:377] Logged in from keytab as oryx/127.20.81.129@KRBTEST.COM (short username oryx)
I20260810 00:58:21.691653 27902 webserver.cc:460] Webserver started at http://127.20.81.129:38657/ using document root <none> and password file <none>
I20260810 00:58:21.692329 27902 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:21.692512 27902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:21.692819 27902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:21.695919 27902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "64786d2c70b344f8954cb3344937e3b7"
format_stamp: "Formatted at 2026-08-10 00:58:21 on dist-test-slave-gp6n"
server_key: "920e036602a9da6ecf45fc0d8d077a05"
server_key_iv: "1bd426dd64a0b693fc82b75a5dfea0e5"
server_key_version: "encryptionkey@0"
I20260810 00:58:21.696715 27902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "64786d2c70b344f8954cb3344937e3b7"
format_stamp: "Formatted at 2026-08-10 00:58:21 on dist-test-slave-gp6n"
server_key: "920e036602a9da6ecf45fc0d8d077a05"
server_key_iv: "1bd426dd64a0b693fc82b75a5dfea0e5"
server_key_version: "encryptionkey@0"
I20260810 00:58:21.701918 27902 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 00:58:21.705325 27919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:21.706805 27902 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:58:21.706976 27902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "64786d2c70b344f8954cb3344937e3b7"
format_stamp: "Formatted at 2026-08-10 00:58:21 on dist-test-slave-gp6n"
server_key: "920e036602a9da6ecf45fc0d8d077a05"
server_key_iv: "1bd426dd64a0b693fc82b75a5dfea0e5"
server_key_version: "encryptionkey@0"
I20260810 00:58:21.707180 27902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:21.736011 27902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:21.740425 27902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:21.740716 27902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:21.741670 27902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:21.743033 27902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:21.743136 27902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:21.743278 27902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:21.743366 27902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:21.777313 27902 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:33253
I20260810 00:58:21.777381 28046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:33253 every 8 connection(s)
I20260810 00:58:21.778751 27902 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 00:58:21.786900 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 27902
I20260810 00:58:21.787151 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 00:58:21.787811 20806 external_mini_cluster.cc:1428] Setting key b824294c2883f044e56fd627a72d502f
Aug 10 00:58:21 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323501, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM
I20260810 00:58:21.800792 28049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:21.781235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:49143 (local address 127.20.81.190:33681)
0810 00:58:21.781663 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:21.781669 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:21.782848 (+ 1179us) server_negotiation.cc:407] Connection header received
0810 00:58:21.784787 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:21.784802 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:21.784905 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:21.785029 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:21.786661 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:21.787511 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:21.789420 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:21.789934 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:21.790033 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:21.794927 (+ 4894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:21.794932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:21.794935 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:21.794966 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:21.797561 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:21.798300 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:21.798306 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:21.798308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:21.798477 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:21.798913 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:21.798918 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:21.798921 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:21.799200 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:21.799398 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:21.800175 (+ 777us) server_negotiation.cc:299] Negotiation successful
0810 00:58:21.800442 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":207,"threads_started":1}
I20260810 00:58:21.801878 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:21.781848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.129:49143)
0810 00:58:21.782663 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:21.782710 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:21.784501 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:21.785277 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:21.785286 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:21.785816 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:21.786490 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:21.786511 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:21.787680 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:21.787692 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:21.789205 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:21.789219 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:21.790163 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:21.790170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:21.790579 (+ 409us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:21.791327 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:21.791364 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:21.794705 (+ 3341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:21.797714 (+ 3009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:21.797723 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:21.797739 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:21.798167 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:21.798588 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:21.798593 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:21.798597 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:21.798793 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:21.799346 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:21.799354 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:21.799724 (+ 370us) client_negotiation.cc:769] Sending connection context
0810 00:58:21.800034 (+ 310us) client_negotiation.cc:240] Negotiation successful
0810 00:58:21.800588 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":275,"threads_started":1}
I20260810 00:58:21.803522 28047 heartbeater.cc:347] Connected to a master server at 127.20.81.190:33681
I20260810 00:58:21.803874 28047 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:21.804775 28047 heartbeater.cc:511] Master 127.20.81.190:33681 requested a full tablet report, sending...
I20260810 00:58:21.807292 27838 ts_manager.cc:194] Registered new tserver with Master: 64786d2c70b344f8954cb3344937e3b7 (127.20.81.129:33253)
I20260810 00:58:21.809543 27838 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:49143
WARNING: no policy specified for oryx/127.20.81.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.81.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.81.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.130.keytab.
Entry for principal oryx/127.20.81.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.130.keytab.
I20260810 00:58:21.846741 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:33681
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.130.keytab
--principal=oryx/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:40295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:22.192886 28053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:22.193368 28053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:22.193487 28053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:22.200927 28053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:22.201020 28053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:22.201154 28053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 00:58:22.210871 28053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:40295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.130.keytab
--principal=oryx/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:33681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:22.212056 28053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:22.213325 28053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:22.228585 28058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:22.228662 28059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:22.228618 28062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:22.229492 28053 server_base.cc:1034] running on GCE node
I20260810 00:58:22.230095 28053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:22.231215 28053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:22.232448 28053 hybrid_clock.cc:648] HybridClock initialized: now 1786323502232401 us; error 57 us; skew 500 ppm
I20260810 00:58:22.232798 28053 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:22 dist-test-slave-gp6n krb5kdc[27789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323502, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:22.236171 28053 init.cc:377] Logged in from keytab as oryx/127.20.81.130@KRBTEST.COM (short username oryx)
I20260810 00:58:22.237630 28053 webserver.cc:460] Webserver started at http://127.20.81.130:40595/ using document root <none> and password file <none>
I20260810 00:58:22.238317 28053 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:22.238469 28053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:22.238787 28053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:22.241657 28053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "4b439751b0f64ffd8322d2894db470ac"
format_stamp: "Formatted at 2026-08-10 00:58:22 on dist-test-slave-gp6n"
server_key: "6d1cc4e186c2a68ad0188278a40d1810"
server_key_iv: "223b3701ff5b52c3f975465e49763574"
server_key_version: "encryptionkey@0"
I20260810 00:58:22.242519 28053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "4b439751b0f64ffd8322d2894db470ac"
format_stamp: "Formatted at 2026-08-10 00:58:22 on dist-test-slave-gp6n"
server_key: "6d1cc4e186c2a68ad0188278a40d1810"
server_key_iv: "223b3701ff5b52c3f975465e49763574"
server_key_version: "encryptionkey@0"
I20260810 00:58:22.247725 28053 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260810 00:58:22.251148 28069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:22.252477 28053 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 00:58:22.252647 28053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "4b439751b0f64ffd8322d2894db470ac"
format_stamp: "Formatted at 2026-08-10 00:58:22 on dist-test-slave-gp6n"
server_key: "6d1cc4e186c2a68ad0188278a40d1810"
server_key_iv: "223b3701ff5b52c3f975465e49763574"
server_key_version: "encryptionkey@0"
I20260810 00:58:22.252851 28053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:22.266600 28053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:22.270955 28053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:22.271265 28053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:22.272287 28053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:22.273635 28053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:22.273736 28053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:22.273888 28053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:22.273965 28053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:22.306846 28053 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:36539
I20260810 00:58:22.306885 28198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:36539 every 8 connection(s)
I20260810 00:58:22.308153 28053 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 00:58:22.311589 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 28053
I20260810 00:58:22.311796 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 00:58:22.312242 20806 external_mini_cluster.cc:1428] Setting key 4736eecbace88ca0fa32a8528e27323a
Aug 10 00:58:22 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323502, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM
I20260810 00:58:22.329932 28201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:22.310797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:47523 (local address 127.20.81.190:33681)
0810 00:58:22.311368 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:22.311377 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:22.312499 (+ 1122us) server_negotiation.cc:407] Connection header received
0810 00:58:22.314595 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:22.314602 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:22.314682 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:22.314793 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:22.316239 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:22.317388 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:22.318981 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:22.319742 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:22.319895 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:22.323959 (+ 4064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:22.323964 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:22.323967 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:22.324006 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:22.326902 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:22.327629 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:22.327639 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:22.327641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:22.327766 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:22.328203 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:22.328207 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:22.328209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:22.328458 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:22.328609 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:22.329295 (+ 686us) server_negotiation.cc:299] Negotiation successful
0810 00:58:22.329520 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":280,"threads_started":1}
I20260810 00:58:22.331053 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:22.311456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.130:47523)
0810 00:58:22.312326 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:22.312377 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:22.314293 (+ 1916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:22.315013 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:22.315021 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:22.315505 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:22.316067 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:22.316085 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:22.317547 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:22.317550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:22.318846 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:22.318857 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:22.320036 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:22.320042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:22.320256 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:22.320947 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:22.320982 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:22.323727 (+ 2745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:22.327051 (+ 3324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:22.327060 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:22.327076 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:22.327499 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:22.327877 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:22.327883 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:22.327887 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:22.328075 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:22.328584 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:22.328591 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:22.328934 (+ 343us) client_negotiation.cc:769] Sending connection context
0810 00:58:22.329188 (+ 254us) client_negotiation.cc:240] Negotiation successful
0810 00:58:22.329729 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":363,"threads_started":1}
I20260810 00:58:22.332549 28199 heartbeater.cc:347] Connected to a master server at 127.20.81.190:33681
I20260810 00:58:22.332888 28199 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:22.333606 28199 heartbeater.cc:511] Master 127.20.81.190:33681 requested a full tablet report, sending...
I20260810 00:58:22.335167 27838 ts_manager.cc:194] Registered new tserver with Master: 4b439751b0f64ffd8322d2894db470ac (127.20.81.130:36539)
I20260810 00:58:22.336414 27838 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:47523
WARNING: no policy specified for oryx/127.20.81.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.81.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.81.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.131.keytab.
Entry for principal oryx/127.20.81.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.131.keytab.
I20260810 00:58:22.367956 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:33681
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.131.keytab
--principal=oryx/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:40295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:22.700932 28205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:22.701262 28205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:22.701383 28205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:22.708395 28205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:22.708493 28205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:22.708607 28205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 00:58:22.718107 28205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:40295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.131.keytab
--principal=oryx/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:33681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:22.719410 28205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:22.720758 28205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:22.734436 28210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:22.734479 28211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:22.734442 28213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:22.735044 28205 server_base.cc:1034] running on GCE node
I20260810 00:58:22.735931 28205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:22.736991 28205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:22.738251 28205 hybrid_clock.cc:648] HybridClock initialized: now 1786323502738202 us; error 35 us; skew 500 ppm
I20260810 00:58:22.738644 28205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:22 dist-test-slave-gp6n krb5kdc[27789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323502, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:22.742020 28205 init.cc:377] Logged in from keytab as oryx/127.20.81.131@KRBTEST.COM (short username oryx)
I20260810 00:58:22.743695 28205 webserver.cc:460] Webserver started at http://127.20.81.131:41941/ using document root <none> and password file <none>
I20260810 00:58:22.744442 28205 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:22.744604 28205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:22.744952 28205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:22.748219 28205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "94f458e21af3447e8489ca2b0f0c2e07"
format_stamp: "Formatted at 2026-08-10 00:58:22 on dist-test-slave-gp6n"
server_key: "4f8ca886f049127d7331382408bcecf5"
server_key_iv: "9f86909ae809e03cb2a6fcdcc11a3d93"
server_key_version: "encryptionkey@0"
I20260810 00:58:22.749081 28205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "94f458e21af3447e8489ca2b0f0c2e07"
format_stamp: "Formatted at 2026-08-10 00:58:22 on dist-test-slave-gp6n"
server_key: "4f8ca886f049127d7331382408bcecf5"
server_key_iv: "9f86909ae809e03cb2a6fcdcc11a3d93"
server_key_version: "encryptionkey@0"
I20260810 00:58:22.754753 28205 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 00:58:22.758500 28221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:22.760108 28205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 00:58:22.760334 28205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "94f458e21af3447e8489ca2b0f0c2e07"
format_stamp: "Formatted at 2026-08-10 00:58:22 on dist-test-slave-gp6n"
server_key: "4f8ca886f049127d7331382408bcecf5"
server_key_iv: "9f86909ae809e03cb2a6fcdcc11a3d93"
server_key_version: "encryptionkey@0"
I20260810 00:58:22.760584 28205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:22.781203 28205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:22.785909 28205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:22.786191 28205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:22.787258 28205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:22.788759 28205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:22.788838 28205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:22.788937 28205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:22.788987 28205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:22.813650 28047 heartbeater.cc:503] Master 127.20.81.190:33681 was elected leader, sending a full tablet report...
I20260810 00:58:22.826713 28205 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:35525
I20260810 00:58:22.826752 28349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:35525 every 8 connection(s)
I20260810 00:58:22.828135 28205 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 00:58:22.832834 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 28205
I20260810 00:58:22.833057 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 00:58:22.833630 20806 external_mini_cluster.cc:1428] Setting key 65a682acda633857591b120e2296c6df
Aug 10 00:58:22 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323502, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM
I20260810 00:58:22.849931 28352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:22.830981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:47403 (local address 127.20.81.190:33681)
0810 00:58:22.831510 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:22.831515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:22.832682 (+ 1167us) server_negotiation.cc:407] Connection header received
0810 00:58:22.835142 (+ 2460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:22.835148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:22.835248 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:22.835349 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:22.837230 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:22.838093 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:22.839761 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:22.840165 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:22.840270 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:22.843741 (+ 3471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:22.843746 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:22.843749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:22.843781 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:22.846374 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:22.847241 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:22.847245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:22.847247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:22.847364 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:22.847917 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:22.847920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:22.847922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:22.848284 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:22.848465 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:22.849359 (+ 894us) server_negotiation.cc:299] Negotiation successful
0810 00:58:22.849641 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":250,"threads_started":1}
I20260810 00:58:22.851068 28353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:22.831555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.131:47403)
0810 00:58:22.832470 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:22.832527 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:22.834821 (+ 2294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:22.835662 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:22.835670 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:22.836312 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:22.837040 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:22.837071 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:22.838250 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:22.838254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:22.839623 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:22.839636 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:22.840384 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:22.840389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:22.840595 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:22.841321 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:22.841344 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:22.843542 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:22.846573 (+ 3031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:22.846581 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:22.846595 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:22.847093 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:22.847487 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:22.847493 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:22.847497 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:22.847755 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:22.848451 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:22.848460 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:22.848875 (+ 415us) client_negotiation.cc:769] Sending connection context
0810 00:58:22.849197 (+ 322us) client_negotiation.cc:240] Negotiation successful
0810 00:58:22.849793 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":364,"threads_started":1}
I20260810 00:58:22.852425 28350 heartbeater.cc:347] Connected to a master server at 127.20.81.190:33681
I20260810 00:58:22.852749 28350 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:22.853545 28350 heartbeater.cc:511] Master 127.20.81.190:33681 requested a full tablet report, sending...
I20260810 00:58:22.855072 27838 ts_manager.cc:194] Registered new tserver with Master: 94f458e21af3447e8489ca2b0f0c2e07 (127.20.81.131:35525)
I20260810 00:58:22.855948 27838 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:47403
I20260810 00:58:22.861976 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.20.81.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.81.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.81.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.189.keytab.
Entry for principal oryx/127.20.81.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.189.keytab.
I20260810 00:58:22.918268 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.189:46407
--webserver_interface=127.20.81.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.189.keytab
--principal=oryx/127.20.81.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:40295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:33681,127.20.81.189:46407
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:23.251667 28356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:23.251938 28356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:23.252135 28356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:23.259449 28356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:58:23.259548 28356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:23.259662 28356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:23.259764 28356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:58:23.259828 28356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:58:23.269506 28356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:40295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:33681,127.20.81.189:46407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.189.keytab
--principal=oryx/127.20.81.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.189:46407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.81.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
W20260810 00:58:23.269855 28356 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.20.81.190:33681,127.20.81.189:46407'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 00:58:23.270948 28356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:23.272246 28356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:23.284179 28362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:23.284183 28365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:23.285251 28361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:23.285789 28356 server_base.cc:1034] running on GCE node
I20260810 00:58:23.286609 28356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:23.288030 28356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:23.289248 28356 hybrid_clock.cc:648] HybridClock initialized: now 1786323503289197 us; error 36 us; skew 500 ppm
I20260810 00:58:23.289594 28356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:23 dist-test-slave-gp6n krb5kdc[27789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323503, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:23.292876 28356 init.cc:377] Logged in from keytab as oryx/127.20.81.189@KRBTEST.COM (short username oryx)
I20260810 00:58:23.294351 28356 webserver.cc:460] Webserver started at http://127.20.81.189:40371/ using document root <none> and password file <none>
I20260810 00:58:23.295109 28356 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:23.295198 28356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:23.295562 28356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:23.298732 28356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/data/instance:
uuid: "025dbc759c9a4ed3986536b4df6c761d"
format_stamp: "Formatted at 2026-08-10 00:58:23 on dist-test-slave-gp6n"
server_key: "f4f4e441df9e0cb5b0b85d1f7914ec68"
server_key_iv: "fd8b245d3f890da7ff9c592ff7d08379"
server_key_version: "encryptionkey@0"
I20260810 00:58:23.299540 28356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/wal/instance:
uuid: "025dbc759c9a4ed3986536b4df6c761d"
format_stamp: "Formatted at 2026-08-10 00:58:23 on dist-test-slave-gp6n"
server_key: "f4f4e441df9e0cb5b0b85d1f7914ec68"
server_key_iv: "fd8b245d3f890da7ff9c592ff7d08379"
server_key_version: "encryptionkey@0"
I20260810 00:58:23.304994 28356 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 00:58:23.308797 28374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:23.310479 28356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 00:58:23.310674 28356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/wal
uuid: "025dbc759c9a4ed3986536b4df6c761d"
format_stamp: "Formatted at 2026-08-10 00:58:23 on dist-test-slave-gp6n"
server_key: "f4f4e441df9e0cb5b0b85d1f7914ec68"
server_key_iv: "fd8b245d3f890da7ff9c592ff7d08379"
server_key_version: "encryptionkey@0"
I20260810 00:58:23.310922 28356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:23.332935 28356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:23.337059 28356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:23.337337 28356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:23.339032 28199 heartbeater.cc:503] Master 127.20.81.190:33681 was elected leader, sending a full tablet report...
I20260810 00:58:23.356663 28356 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.189:46407
I20260810 00:58:23.356702 28430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.189:46407 every 8 connection(s)
I20260810 00:58:23.358160 28356 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/data/info.pb
I20260810 00:58:23.361958 28431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:23.362182 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 28356
I20260810 00:58:23.372201 28431 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 00:58:23 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.81.189@KRBTEST.COM
Aug 10 00:58:23 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323503, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.189@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM
I20260810 00:58:23.386960 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:23.364379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41410 (local address 127.20.81.189:46407)
0810 00:58:23.365257 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:23.365273 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:23.365326 (+ 53us) server_negotiation.cc:407] Connection header received
0810 00:58:23.367089 (+ 1763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:23.367118 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:23.367633 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:23.368055 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:23.369337 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.371380 (+ 2043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.372728 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.373710 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.373875 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:23.378968 (+ 5093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:23.378974 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:23.378986 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:23.379023 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:23.382817 (+ 3794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:23.383510 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:23.383520 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:23.383536 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:23.383756 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:23.384162 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:23.384170 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:23.384175 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:23.384759 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:23.385074 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:23.385521 (+ 447us) server_negotiation.cc:299] Negotiation successful
0810 00:58:23.385924 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":371,"threads_started":1}
I20260810 00:58:23.390866 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:23.374956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46304 (local address 127.20.81.190:33681)
0810 00:58:23.375450 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:23.375458 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:23.375970 (+ 512us) server_negotiation.cc:407] Connection header received
0810 00:58:23.376162 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:23.376167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:23.376281 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:23.376407 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:23.377891 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.379020 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.381180 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.381572 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.381670 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:23.385122 (+ 3452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:23.385127 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:23.385131 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:23.385164 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:23.387973 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:23.388653 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:23.388658 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:23.388661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:23.388801 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:23.389170 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:23.389175 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:23.389178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:23.389553 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:23.389730 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:23.390261 (+ 531us) server_negotiation.cc:299] Negotiation successful
0810 00:58:23.390523 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":235,"threads_started":1}
I20260810 00:58:23.390905 28436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:23.374957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.0.0.1:46304)
0810 00:58:23.375825 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:23.375866 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:23.376026 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:23.376454 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:23.376458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:23.377325 (+ 867us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:23.377728 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:23.377759 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.379160 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.379164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:23.381033 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:23.381055 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.381751 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.381757 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:23.381941 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:23.382521 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:23.382547 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:23.384932 (+ 2385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:23.388109 (+ 3177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:23.388117 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:23.388124 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:23.388538 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:23.388913 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:23.388916 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:23.388918 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:23.389065 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:23.389830 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:23.389835 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:23.389973 (+ 138us) client_negotiation.cc:769] Sending connection context
0810 00:58:23.390171 (+ 198us) client_negotiation.cc:240] Negotiation successful
0810 00:58:23.390493 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":512,"threads_started":1}
I20260810 00:58:23.392480 28431 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 00:58:23 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323503, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.189@KRBTEST.COM for oryx/127.20.81.189@KRBTEST.COM
I20260810 00:58:23.404467 28436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:23.393168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:46407 (local address 127.0.0.1:41416)
0810 00:58:23.393326 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:23.393345 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:23.393478 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:23.393764 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:23.393768 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:23.394105 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:23.394230 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:23.394237 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.395229 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.395234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:23.396086 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:23.396095 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.396748 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.396753 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:23.396923 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:23.397460 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:23.397476 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:23.399387 (+ 1911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:23.402069 (+ 2682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:23.402074 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:23.402078 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:23.402482 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:23.402834 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:23.402838 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:23.402842 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:23.402966 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:23.403500 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:23.403504 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:23.403637 (+ 133us) client_negotiation.cc:769] Sending connection context
0810 00:58:23.403772 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 00:58:23.404059 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":1}
I20260810 00:58:23.404528 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:23.393132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41416 (local address 127.20.81.189:46407)
0810 00:58:23.393323 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:23.393329 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:23.393418 (+ 89us) server_negotiation.cc:407] Connection header received
0810 00:58:23.393591 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:23.393596 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:23.393654 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:23.393750 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:23.394355 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.395090 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.396227 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.396605 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.396736 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:23.399553 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:23.399563 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:23.399566 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:23.399592 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:23.401904 (+ 2312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:23.402593 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:23.402600 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:23.402603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:23.402713 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:23.403065 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:23.403073 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:23.403076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:23.403363 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:23.403518 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:23.403844 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 00:58:23.404203 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":156,"server-negotiator.queue_time_us":68}
I20260810 00:58:23.409080 28431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d: Bootstrap starting.
I20260810 00:58:23.412252 28431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:23.413256 28431 log.cc:824] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:23.416162 28431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d: No bootstrap required, opened a new log
I20260810 00:58:23.422974 28431 raft_consensus.cc:348] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } }
I20260810 00:58:23.423290 28431 raft_consensus.cc:374] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:23.423385 28431 raft_consensus.cc:729] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025dbc759c9a4ed3986536b4df6c761d, State: Initialized, Role: FOLLOWER
I20260810 00:58:23.423976 28431 consensus_queue.cc:260] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } }
I20260810 00:58:23.424913 28438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } } }
I20260810 00:58:23.425274 28438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:58:23.425885 28431 sys_catalog.cc:564] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [sys.catalog]: configured and running, proceeding with master startup.
W20260810 00:58:23.434641 28449 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 00:58:23.434783 28449 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 00:58:23.437405 28356 master_runner.cc:428] Detected that this master 025dbc759c9a4ed3986536b4df6c761d is joining an existing cluster
I20260810 00:58:23.437517 28356 master_runner.cc:432] Initiating AddMaster RPC to add 127.20.81.189:46407
I20260810 00:58:23.469672 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:23.442515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41432 (local address 127.20.81.189:46407)
0810 00:58:23.442697 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:23.442706 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:23.452796 (+ 10090us) server_negotiation.cc:407] Connection header received
0810 00:58:23.453486 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:23.453492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:23.453615 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:23.453713 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:23.455620 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.456882 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.458447 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.459297 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.459431 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:23.462149 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:23.462158 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:23.462165 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:23.462256 (+ 91us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:23.466237 (+ 3981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:23.467128 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:23.467135 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:23.467141 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:23.467292 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:23.467723 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:23.467730 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:23.467736 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:23.468124 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:23.468262 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:23.468876 (+ 614us) server_negotiation.cc:299] Negotiation successful
0810 00:58:23.469128 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":32,"spinlock_wait_cycles":22784}
I20260810 00:58:23.471127 28456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:23.441852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:46407 (local address 127.0.0.1:41432)
0810 00:58:23.452538 (+ 10686us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:23.452561 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:23.453282 (+ 721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:23.453962 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:23.453969 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:23.455026 (+ 1057us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:23.455448 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:23.455462 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.457135 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.457146 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:23.458286 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:23.458297 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.459711 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.459720 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:23.459951 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:23.460662 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:23.460689 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:23.461953 (+ 1264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:23.466374 (+ 4421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:23.466381 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:23.466388 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:23.467006 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:23.467421 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:23.467427 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:23.467433 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:23.467612 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:23.468330 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:23.468337 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:23.468570 (+ 233us) client_negotiation.cc:769] Sending connection context
0810 00:58:23.470249 (+ 1679us) client_negotiation.cc:240] Negotiation successful
0810 00:58:23.470603 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10502,"spinlock_wait_cycles":73984,"thread_start_us":10961,"threads_started":1}
I20260810 00:58:23.471189 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:23.441600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46306 (local address 127.20.81.190:33681)
0810 00:58:23.441781 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:23.441787 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:23.455234 (+ 13447us) server_negotiation.cc:407] Connection header received
0810 00:58:23.455520 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:23.455528 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:23.455634 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:23.455709 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:23.457440 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.458570 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.460704 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.461543 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.461658 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:23.464221 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:23.464228 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:23.464232 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:23.464280 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:23.467880 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:23.468666 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:23.468671 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:23.468675 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:23.468841 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:23.469356 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:23.469361 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:23.469365 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:23.469791 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:23.469918 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:23.470597 (+ 679us) server_negotiation.cc:299] Negotiation successful
0810 00:58:23.470830 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 00:58:23.472543 28457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:23.442132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.0.0.1:46306)
0810 00:58:23.454932 (+ 12800us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:23.454947 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:23.455355 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:23.456021 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:23.456028 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:23.456813 (+ 785us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:23.457252 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:23.457269 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.458776 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.458781 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:23.460526 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:23.460542 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.461952 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.461958 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:23.462139 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:23.462823 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:23.462845 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:23.463992 (+ 1147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:23.468041 (+ 4049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:23.468046 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:23.468050 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:23.468519 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:23.469018 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:23.469022 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:23.469026 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:23.469197 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:23.470023 (+ 826us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:23.470028 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:23.470261 (+ 233us) client_negotiation.cc:769] Sending connection context
0810 00:58:23.471472 (+ 1211us) client_negotiation.cc:240] Negotiation successful
0810 00:58:23.471979 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12684,"mutex_wait_us":51,"thread_start_us":12469,"threads_started":1}
WARNING: no policy specified for oryx/127.20.81.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.81.188@KRBTEST.COM" created.
Aug 10 00:58:23 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323501, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.190@KRBTEST.COM for oryx/127.20.81.189@KRBTEST.COM
I20260810 00:58:23.509248 28459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:23.479622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:46407 (local address 127.0.0.1:41442)
0810 00:58:23.487027 (+ 7405us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:23.487082 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:23.487537 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:23.488344 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:23.488356 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:23.489322 (+ 966us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:23.489630 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:23.489644 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.491314 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.491322 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:23.492942 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:23.492959 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.494159 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.494167 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:23.494503 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:23.498446 (+ 3943us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:23.498473 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:23.501797 (+ 3324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:23.506226 (+ 4429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:23.506266 (+ 40us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:23.506278 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:23.506843 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:23.507285 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:23.507290 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:23.507296 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:23.507468 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:23.508218 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:23.508225 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:23.508436 (+ 211us) client_negotiation.cc:769] Sending connection context
0810 00:58:23.508591 (+ 155us) client_negotiation.cc:240] Negotiation successful
0810 00:58:23.508845 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7118,"thread_start_us":6955,"threads_started":1}
I20260810 00:58:23.509358 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:23.479627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41442 (local address 127.20.81.189:46407)
0810 00:58:23.479807 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:23.479812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:23.487413 (+ 7601us) server_negotiation.cc:407] Connection header received
0810 00:58:23.487809 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:23.487815 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:23.487933 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:23.488046 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:23.489836 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.491140 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.493122 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.493974 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.494119 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:23.502044 (+ 7925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:23.502051 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:23.502056 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:23.502101 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:23.506034 (+ 3933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:23.506967 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:23.506972 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:23.506976 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:23.507142 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:23.507602 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:23.507608 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:23.507611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:23.508072 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:23.508226 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:23.508758 (+ 532us) server_negotiation.cc:299] Negotiation successful
0810 00:58:23.508990 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.81.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.188.keytab.
Entry for principal oryx/127.20.81.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.188.keytab.
I20260810 00:58:23.510859 28438 raft_consensus.cc:482] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:58:23.511170 28438 raft_consensus.cc:504] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } }
I20260810 00:58:23.511706 27838 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.20.81.189:46407: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d"
member_type: NON_VOTER
last_known_addr {
host: "127.20.81.189"
port: 46407
}
attrs {
promote: true
}
}
dest_uuid: "31ff0eea6672410291c03a7df56d1cf2"
cas_config_opid_index: -1
I20260810 00:58:23.512614 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.188:45483
--webserver_interface=127.20.81.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.188.keytab
--principal=oryx/127.20.81.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:40295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:33681,127.20.81.189:46407,127.20.81.188:45483
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:58:23.512796 28438 leader_election.cc:288] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31ff0eea6672410291c03a7df56d1cf2 (127.20.81.190:33681)
I20260810 00:58:23.513065 27838 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: NON_VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: true } }
I20260810 00:58:23.513468 27848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "025dbc759c9a4ed3986536b4df6c761d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ff0eea6672410291c03a7df56d1cf2" is_pre_election: true
I20260810 00:58:23.514660 28379 leader_election.cc:302] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 025dbc759c9a4ed3986536b4df6c761d; no voters: 31ff0eea6672410291c03a7df56d1cf2
I20260810 00:58:23.515442 28438 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 00:58:23.526230 28462 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 025dbc759c9a4ed3986536b4df6c761d (127.20.81.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: NON_VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: true } } }
I20260810 00:58:23.526990 28403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:23.531112 28462 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.20.81.189:46407
I20260810 00:58:23.531205 28403 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 1 FOLLOWER]: Refusing update from remote peer 31ff0eea6672410291c03a7df56d1cf2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 00:58:23.531740 28463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "31ff0eea6672410291c03a7df56d1cf2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: NON_VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: true } } }
I20260810 00:58:23.532068 28463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:23.532447 28356 master.cc:537] Master@127.20.81.189:46407 shutting down...
I20260810 00:58:23.534185 28462 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: NON_VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:58:23.552911 28356 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:58:23.553690 28356 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:23.554374 28356 tablet_replica.cc:330] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d: stopping tablet replica
I20260810 00:58:23.567019 28356 master.cc:552] Master@127.20.81.189:46407 shutdown complete.
I20260810 00:58:23.567545 28356 master_runner.cc:305] Clearing existing system tablet
I20260810 00:58:23.568269 28356 fs_manager.cc:359] Metadata directory not provided
W20260810 00:58:23.572898 27822 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from unknown peer (error 108)
I20260810 00:58:23.573707 28356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:23.578473 28459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:23.577918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:46407 (local address 127.0.0.1:41452)
0810 00:58:23.578081 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:23.578240 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:46407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 00:58:23.578684 27822 consensus_peers.cc:487] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 -> Peer 025dbc759c9a4ed3986536b4df6c761d (127.20.81.189:46407): Couldn't send request to peer 025dbc759c9a4ed3986536b4df6c761d. Status: Network error: Client connection negotiation failed: client connection to 127.20.81.189:46407: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 00:58:23.579540 28356 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.001s
I20260810 00:58:23.581274 28471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:23.581677 28356 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:23.581996 28356 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260810 00:58:23.582085 28356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/wal
uuid: "025dbc759c9a4ed3986536b4df6c761d"
format_stamp: "Formatted at 2026-08-10 00:58:23 on dist-test-slave-gp6n"
server_key: "f4f4e441df9e0cb5b0b85d1f7914ec68"
server_key_iv: "fd8b245d3f890da7ff9c592ff7d08379"
server_key_version: "encryptionkey@0"
I20260810 00:58:23.586092 28356 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:23.590888 28356 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 00:58:23.591073 28356 log.cc:1196] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 00:58:23.591508 28356 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d: Deleting consensus metadata
I20260810 00:58:23.591773 28356 master_runner.cc:315] Copying system tablet from 127.20.81.190:33681
I20260810 00:58:23.594237 28356 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d: tablet copy: Beginning tablet copy session from remote peer at address 127.20.81.190:33681
I20260810 00:58:23.613855 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:23.595043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46322 (local address 127.20.81.190:33681)
0810 00:58:23.595274 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:23.595283 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:23.598948 (+ 3665us) server_negotiation.cc:407] Connection header received
0810 00:58:23.599217 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:23.599222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:23.599291 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:23.599410 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:23.600410 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.601246 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.602525 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.602982 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.603138 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:23.608130 (+ 4992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:23.608136 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:23.608139 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:23.608171 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:23.611079 (+ 2908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:23.611811 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:23.611817 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:23.611820 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:23.611961 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:23.612398 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:23.612402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:23.612406 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:23.612684 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:23.612844 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:23.613254 (+ 410us) server_negotiation.cc:299] Negotiation successful
0810 00:58:23.613459 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 00:58:23.614007 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:23.594990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.0.0.1:46322)
0810 00:58:23.598817 (+ 3827us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:23.598835 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:23.599023 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:23.599458 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:23.599464 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:23.599934 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:23.600250 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:23.600264 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.601413 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.601418 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:23.602328 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:23.602343 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:23.603166 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:23.603171 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:23.603382 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:23.607176 (+ 3794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:23.607194 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:23.607916 (+ 722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:23.611255 (+ 3339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:23.611260 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:23.611264 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:23.611620 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:23.612119 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:23.612126 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:23.612129 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:23.612279 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:23.612809 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:23.612814 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:23.612975 (+ 161us) client_negotiation.cc:769] Sending connection context
0810 00:58:23.613136 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 00:58:23.613451 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3686,"thread_start_us":3590,"threads_started":1}
I20260810 00:58:23.614930 27858 tablet_copy_service.cc:140] P 31ff0eea6672410291c03a7df56d1cf2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 025dbc759c9a4ed3986536b4df6c761d ({username='oryx', principal='oryx/127.20.81.189@KRBTEST.COM'} at 127.0.0.1:46322)
I20260810 00:58:23.615172 27858 tablet_copy_service.cc:161] P 31ff0eea6672410291c03a7df56d1cf2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 025dbc759c9a4ed3986536b4df6c761d at {username='oryx', principal='oryx/127.20.81.189@KRBTEST.COM'} at 127.0.0.1:46322: session id = 025dbc759c9a4ed3986536b4df6c761d-00000000000000000000000000000000
I20260810 00:58:23.618155 27858 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 00:58:23.620060 28356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 00:58:23.624954 28356 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d: tablet copy: Starting download of 0 data blocks...
I20260810 00:58:23.625382 28356 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d: tablet copy: Starting download of 1 WAL segments...
I20260810 00:58:23.627903 28356 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 00:58:23.631448 27858 tablet_copy_service.cc:342] P 31ff0eea6672410291c03a7df56d1cf2: Request end of tablet copy session 025dbc759c9a4ed3986536b4df6c761d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.20.81.189@KRBTEST.COM'} at 127.0.0.1:46322
I20260810 00:58:23.631633 27858 tablet_copy_service.cc:433] P 31ff0eea6672410291c03a7df56d1cf2: ending tablet copy session 025dbc759c9a4ed3986536b4df6c761d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 025dbc759c9a4ed3986536b4df6c761d
I20260810 00:58:23.641928 28356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:23.644501 28479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:23.650612 28480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:23.657320 28482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:23.657820 28356 server_base.cc:1034] running on GCE node
I20260810 00:58:23.658200 28356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:23.658897 28356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:23.660261 28356 hybrid_clock.cc:648] HybridClock initialized: now 1786323503660193 us; error 43 us; skew 500 ppm
I20260810 00:58:23.660486 28356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:23 dist-test-slave-gp6n krb5kdc[27789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323503, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:23.662710 28356 init.cc:377] Logged in from keytab as oryx/127.20.81.189@KRBTEST.COM (short username oryx)
I20260810 00:58:23.667450 28356 webserver.cc:460] Webserver started at http://127.20.81.189:35229/ using document root <none> and password file <none>
I20260810 00:58:23.667905 28356 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:23.668004 28356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:23.671192 28356 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260810 00:58:23.673305 28490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:23.673724 28356 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260810 00:58:23.673851 28356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/wal
uuid: "025dbc759c9a4ed3986536b4df6c761d"
format_stamp: "Formatted at 2026-08-10 00:58:23 on dist-test-slave-gp6n"
server_key: "f4f4e441df9e0cb5b0b85d1f7914ec68"
server_key_iv: "fd8b245d3f890da7ff9c592ff7d08379"
server_key_version: "encryptionkey@0"
I20260810 00:58:23.674018 28356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:23.687968 28356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:23.692407 28356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:23.720814 28356 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.189:46407
I20260810 00:58:23.720844 28547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.189:46407 every 8 connection(s)
I20260810 00:58:23.722012 28356 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-1/data/info.pb
I20260810 00:58:23.727743 28548 sys_catalog.cc:263] Verifying existing consensus state
I20260810 00:58:23.731684 28548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d: Bootstrap starting.
I20260810 00:58:23.748117 28548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 00:58:23.748644 28548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d: Bootstrap complete.
I20260810 00:58:23.749802 28548 raft_consensus.cc:348] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: NON_VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: true } }
I20260810 00:58:23.750095 28548 raft_consensus.cc:729] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 025dbc759c9a4ed3986536b4df6c761d, State: Initialized, Role: LEARNER
I20260810 00:58:23.750540 28548 consensus_queue.cc:260] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: NON_VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: true } }
I20260810 00:58:23.751379 28552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: NON_VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: true } } }
I20260810 00:58:23.751659 28548 sys_catalog.cc:564] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:58:23.751760 28552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [sys.catalog]: This master's current role is: LEARNER
I20260810 00:58:23.767536 28567 catalog_manager.cc:1259] Loaded cluster ID: 8c10a06edd3847a3bd451ed9039cadf6
I20260810 00:58:23.767645 28567 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d: loading cluster ID for follower catalog manager: success
I20260810 00:58:23.771354 28567 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d: acquiring CA information for follower catalog manager: success
I20260810 00:58:23.774140 28567 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 00:58:23.858769 28350 heartbeater.cc:503] Master 127.20.81.190:33681 was elected leader, sending a full tablet report...
W20260810 00:58:23.957621 28465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:23.957974 28465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:23.958132 28465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:23.965838 28465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:58:23.965981 28465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:23.966082 28465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:23.966185 28465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:58:23.966262 28465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:58:23.976547 28465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:40295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:33681,127.20.81.189:46407,127.20.81.188:45483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.188.keytab
--principal=oryx/127.20.81.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.188:45483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.81.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:23.977975 28465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:23.979297 28465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:23.991237 28572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:23.991335 28573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:23.991281 28575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:23.992411 28465 server_base.cc:1034] running on GCE node
I20260810 00:58:23.993185 28465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:23.994650 28465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:23.995939 28465 hybrid_clock.cc:648] HybridClock initialized: now 1786323503995870 us; error 47 us; skew 500 ppm
I20260810 00:58:23.996318 28465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:23 dist-test-slave-gp6n krb5kdc[27789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323503, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:23.999673 28465 init.cc:377] Logged in from keytab as oryx/127.20.81.188@KRBTEST.COM (short username oryx)
I20260810 00:58:24.001199 28465 webserver.cc:460] Webserver started at http://127.20.81.188:43721/ using document root <none> and password file <none>
I20260810 00:58:24.001875 28465 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:24.002022 28465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:24.002324 28465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:24.005257 28465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/data/instance:
uuid: "6f8bda17b7544088b413ba1a962fdd6d"
format_stamp: "Formatted at 2026-08-10 00:58:24 on dist-test-slave-gp6n"
server_key: "81217c8951a6c35f2e3257356ec6f109"
server_key_iv: "76ffb5d12228f3b1a08c94ee336d3a62"
server_key_version: "encryptionkey@0"
I20260810 00:58:24.006037 28465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/wal/instance:
uuid: "6f8bda17b7544088b413ba1a962fdd6d"
format_stamp: "Formatted at 2026-08-10 00:58:24 on dist-test-slave-gp6n"
server_key: "81217c8951a6c35f2e3257356ec6f109"
server_key_iv: "76ffb5d12228f3b1a08c94ee336d3a62"
server_key_version: "encryptionkey@0"
I20260810 00:58:24.011365 28465 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 00:58:24.014740 28582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:24.016139 28465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 00:58:24.016371 28465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/wal
uuid: "6f8bda17b7544088b413ba1a962fdd6d"
format_stamp: "Formatted at 2026-08-10 00:58:24 on dist-test-slave-gp6n"
server_key: "81217c8951a6c35f2e3257356ec6f109"
server_key_iv: "76ffb5d12228f3b1a08c94ee336d3a62"
server_key_version: "encryptionkey@0"
I20260810 00:58:24.016587 28465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:24.037717 28465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:24.041663 28465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:24.041958 28465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:24.067181 28465 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.188:45483
I20260810 00:58:24.067206 28637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.188:45483 every 8 connection(s)
I20260810 00:58:24.068673 28465 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/data/info.pb
I20260810 00:58:24.071928 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 28465
I20260810 00:58:24.072652 28638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:24.082394 28638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 00:58:24 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.81.188@KRBTEST.COM
Aug 10 00:58:24 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323503, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.188@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM
I20260810 00:58:24.098274 28643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:24.074178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48540 (local address 127.20.81.188:45483)
0810 00:58:24.075021 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:24.075034 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:24.075072 (+ 38us) server_negotiation.cc:407] Connection header received
0810 00:58:24.077557 (+ 2485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:24.077577 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:24.078033 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:24.078604 (+ 571us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:24.080262 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.081566 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.083180 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.083981 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.084177 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:24.089203 (+ 5026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:24.089210 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:24.089219 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:24.089257 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:24.093067 (+ 3810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:24.094255 (+ 1188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:24.094268 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:24.094316 (+ 48us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:24.094604 (+ 288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:24.095219 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:24.095231 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:24.095238 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:24.095860 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:24.096232 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:24.096719 (+ 487us) server_negotiation.cc:299] Negotiation successful
0810 00:58:24.097146 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":387,"threads_started":1}
I20260810 00:58:24.102303 28646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:24.092430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:46407 (local address 127.0.0.1:41462)
0810 00:58:24.093010 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:24.093033 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:24.093306 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:24.095377 (+ 2071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:24.095382 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:24.095410 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:24.095690 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:24.095707 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.097525 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.097531 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:24.099592 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:24.099604 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.101304 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.101312 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:24.101531 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:24.101595 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 00:58:24.101762 (+ 167us) client_negotiation.cc:240] Negotiation successful
0810 00:58:24.101910 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":701,"threads_started":1}
I20260810 00:58:24.102461 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:24.085153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46326 (local address 127.20.81.190:33681)
0810 00:58:24.085413 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:24.085421 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:24.086211 (+ 790us) server_negotiation.cc:407] Connection header received
0810 00:58:24.086295 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:24.086302 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:24.086428 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:24.086556 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:24.087572 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.088421 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.089940 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.090779 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.090956 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:24.094967 (+ 4011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:24.094974 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:24.094978 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:24.095018 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:24.098905 (+ 3887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:24.099995 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:24.100001 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:24.100005 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:24.100191 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:24.100699 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:24.100706 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:24.100710 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:24.101078 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:24.101245 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:24.101776 (+ 531us) server_negotiation.cc:299] Negotiation successful
0810 00:58:24.102012 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 00:58:24.102514 28644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:24.085082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.0.0.1:46326)
0810 00:58:24.085900 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:24.085942 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:24.086098 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:24.086600 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:24.086604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:24.087165 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:24.087406 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:24.087415 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.088565 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.088570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:24.089767 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:24.089780 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.091020 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.091027 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:24.091214 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:24.091898 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:24.091920 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:24.094717 (+ 2797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:24.099074 (+ 4357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:24.099084 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:24.099091 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:24.099778 (+ 687us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:24.100363 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:24.100367 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:24.100369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:24.100577 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:24.101229 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:24.101237 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:24.101443 (+ 206us) client_negotiation.cc:769] Sending connection context
0810 00:58:24.101651 (+ 208us) client_negotiation.cc:240] Negotiation successful
0810 00:58:24.102009 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":442,"threads_started":1}
I20260810 00:58:24.103510 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:24.092897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41462 (local address 127.20.81.189:46407)
0810 00:58:24.094889 (+ 1992us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:24.094896 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:24.094912 (+ 16us) server_negotiation.cc:407] Connection header received
0810 00:58:24.094981 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:24.094986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:24.095055 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:24.095322 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:24.095866 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.097375 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.099753 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.101073 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.102750 (+ 1677us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:24.102946 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:24.103023 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 00:58:24.103169 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1836,"thread_start_us":1709,"threads_started":1}
I20260810 00:58:24.104424 28638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } has no permanent_uuid. Determining permanent_uuid...
I20260810 00:58:24.104776 28520 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260810 00:58:24.105432 28552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31ff0eea6672410291c03a7df56d1cf2. Latest consensus state: current_term: 1 leader_uuid: "31ff0eea6672410291c03a7df56d1cf2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: NON_VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: true } } }
I20260810 00:58:24.105710 28552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [sys.catalog]: This master's current role is: LEARNER
Aug 10 00:58:24 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323503, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.188@KRBTEST.COM for oryx/127.20.81.189@KRBTEST.COM
I20260810 00:58:24.117729 28644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:24.105499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:46407 (local address 127.0.0.1:41472)
0810 00:58:24.105742 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:24.105766 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:24.105878 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:24.106429 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:24.106465 (+ 36us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:24.106867 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:24.107009 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:24.107018 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.108001 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.108005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:24.108989 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:24.109001 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.109697 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.109702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:24.109823 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:24.110466 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:24.110484 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:24.112451 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:24.115407 (+ 2956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:24.115412 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:24.115416 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:24.115782 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:24.116163 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:24.116168 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:24.116171 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:24.116307 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:24.116818 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:24.116822 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:24.116971 (+ 149us) client_negotiation.cc:769] Sending connection context
0810 00:58:24.117106 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 00:58:24.117391 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 00:58:24.117741 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:24.105954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41472 (local address 127.20.81.189:46407)
0810 00:58:24.106108 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:24.106112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:24.106127 (+ 15us) server_negotiation.cc:407] Connection header received
0810 00:58:24.106185 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:24.106192 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:24.106268 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:24.106384 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:24.107139 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.107873 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.109126 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.109528 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.109620 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:24.112634 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:24.112639 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:24.112642 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:24.112672 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:24.115239 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:24.115906 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:24.115912 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:24.115916 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:24.116051 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:24.116416 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:24.116423 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:24.116426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:24.116688 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:24.116841 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:24.117197 (+ 356us) server_negotiation.cc:299] Negotiation successful
0810 00:58:24.117430 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 00:58:24.119031 28638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 00:58:24 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323503, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.188@KRBTEST.COM for oryx/127.20.81.188@KRBTEST.COM
I20260810 00:58:24.130828 28644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:24.119606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:45483 (local address 127.0.0.1:48544)
0810 00:58:24.119820 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:24.119839 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:24.119969 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:24.120285 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:24.120289 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:24.120659 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:24.120791 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:24.120798 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.121801 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.121809 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:24.122670 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:24.122681 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.123365 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.123372 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:24.123568 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:24.124109 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:24.124124 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:24.126128 (+ 2004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:24.128831 (+ 2703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:24.128834 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:24.128837 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:24.129176 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:24.129519 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:24.129522 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:24.129524 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:24.129661 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:24.130090 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:24.130093 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:24.130210 (+ 117us) client_negotiation.cc:769] Sending connection context
0810 00:58:24.130310 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 00:58:24.130552 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 00:58:24.131093 28643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:24.119661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48544 (local address 127.20.81.188:45483)
0810 00:58:24.119821 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:24.119827 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:24.119912 (+ 85us) server_negotiation.cc:407] Connection header received
0810 00:58:24.120097 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:24.120102 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:24.120168 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:24.120273 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:24.120917 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.121685 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.122800 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.123201 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.123327 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:24.126281 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:24.126285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:24.126288 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:24.126315 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:24.128713 (+ 2398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:24.129270 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:24.129285 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:24.129287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:24.129426 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:24.129747 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:24.129750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:24.129752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:24.129998 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:24.130101 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:24.130475 (+ 374us) server_negotiation.cc:299] Negotiation successful
0810 00:58:24.130730 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":51}
I20260810 00:58:24.135555 28638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d: Bootstrap starting.
I20260810 00:58:24.138715 28638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:24.139752 28638 log.cc:824] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:24.142448 28638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d: No bootstrap required, opened a new log
I20260810 00:58:24.157142 28638 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } }
I20260810 00:58:24.157816 28638 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:24.158027 28638 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8bda17b7544088b413ba1a962fdd6d, State: Initialized, Role: FOLLOWER
I20260810 00:58:24.158937 28638 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } }
I20260810 00:58:24.160218 28649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } } }
I20260810 00:58:24.160655 28649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:58:24.161767 28638 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [sys.catalog]: configured and running, proceeding with master startup.
W20260810 00:58:24.183012 28660 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 00:58:24.183411 28660 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 00:58:24.189473 28465 master_runner.cc:428] Detected that this master 6f8bda17b7544088b413ba1a962fdd6d is joining an existing cluster
I20260810 00:58:24.189615 28465 master_runner.cc:432] Initiating AddMaster RPC to add 127.20.81.188:45483
I20260810 00:58:24.205943 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 27902
I20260810 00:58:24.225884 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:24.195184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41488 (local address 127.20.81.189:46407)
0810 00:58:24.195378 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:24.195386 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:24.196727 (+ 1341us) server_negotiation.cc:407] Connection header received
0810 00:58:24.197010 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:24.197019 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:24.197130 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:24.197264 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:24.198659 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.199815 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.203160 (+ 3345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.203952 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.204057 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:24.208156 (+ 4099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:24.208166 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:24.208171 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:24.208214 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:24.213163 (+ 4949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:24.214201 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:24.214207 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:24.214211 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:24.214531 (+ 320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:24.215168 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:24.215175 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:24.215179 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:24.215693 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:24.215877 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:24.224367 (+ 8490us) server_negotiation.cc:299] Negotiation successful
0810 00:58:24.224969 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 00:58:24.229405 28666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:24.195667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:46407 (local address 127.0.0.1:41488)
0810 00:58:24.196618 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:24.196641 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:24.196838 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:24.197313 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:24.197318 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:24.198059 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:24.198481 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:24.198494 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.200573 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.200579 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:24.202993 (+ 2414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:24.203007 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.204514 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.204535 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:24.204856 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:24.206103 (+ 1247us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:24.206131 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:24.207864 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:24.213327 (+ 5463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:24.213333 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:24.213337 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:24.213911 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:24.214668 (+ 757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:24.214677 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:24.214681 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:24.215003 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:24.215980 (+ 977us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:24.215987 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:24.220728 (+ 4741us) client_negotiation.cc:769] Sending connection context
0810 00:58:24.227168 (+ 6440us) client_negotiation.cc:240] Negotiation successful
0810 00:58:24.228237 (+ 1069us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"mutex_wait_us":4287,"thread_start_us":624,"threads_started":1}
Aug 10 00:58:24 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.81.129@KRBTEST.COM
I20260810 00:58:24.240708 28668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:24.199333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:45483 (local address 127.0.0.1:48546)
0810 00:58:24.205805 (+ 6472us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:24.205847 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:24.206184 (+ 337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:24.206765 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:24.206781 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:24.207912 (+ 1131us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:24.208663 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:24.208677 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.215059 (+ 6382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.215065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:24.223144 (+ 8079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:24.223161 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.228710 (+ 5549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.228735 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:24.228965 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:24.229794 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:24.229821 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:24.231626 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:24.236649 (+ 5023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:24.236653 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:24.236657 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:24.237348 (+ 691us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:24.237820 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:24.237824 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:24.237828 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:24.238034 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:24.239136 (+ 1102us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:24.239141 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:24.239325 (+ 184us) client_negotiation.cc:769] Sending connection context
0810 00:58:24.239499 (+ 174us) client_negotiation.cc:240] Negotiation successful
0810 00:58:24.239875 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3127,"mutex_wait_us":97,"thread_start_us":2223,"threads_started":1}
I20260810 00:58:24.241456 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:24.195030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46336 (local address 127.20.81.190:33681)
0810 00:58:24.195238 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:24.195246 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:24.199377 (+ 4131us) server_negotiation.cc:407] Connection header received
0810 00:58:24.199717 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:24.199726 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:24.199834 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:24.199928 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:24.203693 (+ 3765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.205225 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.216697 (+ 11472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.217995 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.218155 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:24.231341 (+ 13186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:24.231350 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:24.231356 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:24.231416 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:24.236583 (+ 5167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:24.237585 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:24.237594 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:24.237600 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:24.237859 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:24.238685 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:24.238693 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:24.238696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:24.239469 (+ 773us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:24.239639 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:24.240671 (+ 1032us) server_negotiation.cc:299] Negotiation successful
0810 00:58:24.240958 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 00:58:24.241734 28643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:24.200152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48546 (local address 127.20.81.188:45483)
0810 00:58:24.200361 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:24.200367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:24.205919 (+ 5552us) server_negotiation.cc:407] Connection header received
0810 00:58:24.206324 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:24.206330 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:24.206437 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:24.206506 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:24.208821 (+ 2315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.214740 (+ 5919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.225427 (+ 10687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.228125 (+ 2698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.228272 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:24.231847 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:24.231854 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:24.231858 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:24.231894 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:24.236503 (+ 4609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:24.237495 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:24.237500 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:24.237504 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:24.237672 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:24.238176 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:24.238181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:24.238184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:24.238880 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:24.239066 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:24.241051 (+ 1985us) server_negotiation.cc:299] Negotiation successful
0810 00:58:24.241291 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 00:58:24.244036 28667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:24.195730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.0.0.1:46336)
0810 00:58:24.199284 (+ 3554us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:24.199302 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:24.199514 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:24.200279 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:24.200284 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:24.202212 (+ 1928us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:24.203539 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:24.203550 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.205459 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.205464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:24.216518 (+ 11054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:24.216532 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.218322 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.218330 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:24.218667 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:24.229032 (+ 10365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:24.229060 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:24.231021 (+ 1961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:24.236779 (+ 5758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:24.236801 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:24.236805 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:24.237423 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:24.238036 (+ 613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:24.238045 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:24.238048 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:24.238340 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:24.239743 (+ 1403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:24.239748 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:24.240079 (+ 331us) client_negotiation.cc:769] Sending connection context
0810 00:58:24.241758 (+ 1679us) client_negotiation.cc:240] Negotiation successful
0810 00:58:24.242236 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3401,"mutex_wait_us":2087,"thread_start_us":3096,"threads_started":1}
I20260810 00:58:24.248759 28669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:24.208099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55066 (local address 127.20.81.129:33253)
0810 00:58:24.210908 (+ 2809us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:24.210918 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:24.210967 (+ 49us) server_negotiation.cc:407] Connection header received
0810 00:58:24.211053 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:24.211072 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:24.211486 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:24.220758 (+ 9272us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:24.221185 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.223157 (+ 1972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.225966 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.227212 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.227441 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:24.238640 (+ 11199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:24.238660 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:24.238679 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:24.238723 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:24.244672 (+ 5949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:24.245565 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:24.245576 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:24.245579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:24.245814 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:24.246338 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:24.246343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:24.246346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:24.246863 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:24.247218 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:24.247733 (+ 515us) server_negotiation.cc:299] Negotiation successful
0810 00:58:24.248124 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2526,"thread_start_us":2203,"threads_started":1}
I20260810 00:58:24.250437 28041 generic_service.cc:196] Checking for leaks (request via RPC)
Aug 10 00:58:24 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323501, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.190@KRBTEST.COM for oryx/127.20.81.188@KRBTEST.COM
I20260810 00:58:24.273519 28646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:24.250973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:45483 (local address 127.0.0.1:48550)
0810 00:58:24.251196 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:24.251218 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:24.251362 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:24.252039 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:24.252044 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:24.252762 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:24.253139 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:24.253155 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.255087 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.255094 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:24.256961 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:24.256977 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.258257 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.258265 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:24.258575 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:24.259379 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:24.259407 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:24.263220 (+ 3813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:24.269511 (+ 6291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:24.269517 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:24.269521 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:24.270305 (+ 784us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:24.270819 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:24.270824 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:24.270828 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:24.271245 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:24.272206 (+ 961us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:24.272214 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:24.272507 (+ 293us) client_negotiation.cc:769] Sending connection context
0810 00:58:24.272671 (+ 164us) client_negotiation.cc:240] Negotiation successful
0810 00:58:24.273021 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 00:58:24.274966 28643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:24.251447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48550 (local address 127.20.81.188:45483)
0810 00:58:24.251631 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:24.251649 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:24.251670 (+ 21us) server_negotiation.cc:407] Connection header received
0810 00:58:24.251768 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:24.251773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:24.251865 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:24.253330 (+ 1465us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:24.253586 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.254959 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.257138 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.258044 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.258198 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:24.263491 (+ 5293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:24.263500 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:24.263507 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:24.263555 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:24.269340 (+ 5785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:24.270516 (+ 1176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:24.270521 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:24.270525 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:24.270680 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:24.271518 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:24.271529 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:24.271532 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:24.272014 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:24.273957 (+ 1943us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:24.274089 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 00:58:24.274356 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 00:58:24.276894 27838 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.20.81.188:45483: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d"
member_type: NON_VOTER
last_known_addr {
host: "127.20.81.188"
port: 45483
}
attrs {
promote: true
}
}
dest_uuid: "31ff0eea6672410291c03a7df56d1cf2"
cas_config_opid_index: 5
I20260810 00:58:24.278301 27838 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: NON_VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: true } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: NON_VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: true } }
I20260810 00:58:24.280483 28645 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6f8bda17b7544088b413ba1a962fdd6d (127.20.81.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: NON_VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: true } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: NON_VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: true } } }
I20260810 00:58:24.282478 28611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:24.288462 28611 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 1 FOLLOWER]: Refusing update from remote peer 31ff0eea6672410291c03a7df56d1cf2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20260810 00:58:24.294277 28589 tablet.cc:2369] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:58:24.296131 28645 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.20.81.188:45483
I20260810 00:58:24.299796 28465 master.cc:537] Master@127.20.81.188:45483 shutting down...
I20260810 00:58:24.299587 28673 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: NON_VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.002s
I20260810 00:58:24.302160 28520 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 1 LEARNER]: Refusing update from remote peer 31ff0eea6672410291c03a7df56d1cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 00:58:24.303174 28672 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: NON_VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260810 00:58:24.316258 28671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "31ff0eea6672410291c03a7df56d1cf2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: NON_VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: true } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: NON_VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: true } } }
I20260810 00:58:24.317230 28671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:24.326538 28520 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6f8bda17b7544088b413ba1a962fdd6d (127.20.81.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: NON_VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: true } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: NON_VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: true } } }
W20260810 00:58:24.333004 27825 consensus_peers.cc:487] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 -> Peer 6f8bda17b7544088b413ba1a962fdd6d (127.20.81.188:45483): Couldn't send request to peer 6f8bda17b7544088b413ba1a962fdd6d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 00:58:24.334642 28552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31ff0eea6672410291c03a7df56d1cf2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: NON_VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: true } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: NON_VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: true } } }
I20260810 00:58:24.335170 28552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [sys.catalog]: This master's current role is: LEARNER
I20260810 00:58:24.335357 28671 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2: attempting to promote NON_VOTER 025dbc759c9a4ed3986536b4df6c761d to VOTER
I20260810 00:58:24.336410 28671 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: NON_VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: true } }
I20260810 00:58:24.339469 28520 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 1 LEARNER]: Refusing update from remote peer 31ff0eea6672410291c03a7df56d1cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 00:58:24.340304 28671 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260810 00:58:24.342972 27825 consensus_peers.cc:487] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 -> Peer 6f8bda17b7544088b413ba1a962fdd6d (127.20.81.188:45483): Couldn't send request to peer 6f8bda17b7544088b413ba1a962fdd6d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 00:58:24.343750 28465 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:58:24.344220 28465 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:24.343722 28672 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 025dbc759c9a4ed3986536b4df6c761d (127.20.81.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: NON_VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: true } } }
I20260810 00:58:24.344448 28465 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d: stopping tablet replica
I20260810 00:58:24.349637 28519 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 025dbc759c9a4ed3986536b4df6c761d (127.20.81.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: NON_VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: true } } }
I20260810 00:58:24.350772 28645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "31ff0eea6672410291c03a7df56d1cf2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: NON_VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: true } } }
I20260810 00:58:24.351320 28645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:24.351979 28645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31ff0eea6672410291c03a7df56d1cf2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: NON_VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: true } } }
I20260810 00:58:24.352362 28645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:24.354936 28552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31ff0eea6672410291c03a7df56d1cf2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: NON_VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: true } } }
I20260810 00:58:24.355230 28552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:58:24.365275 28465 master.cc:552] Master@127.20.81.188:45483 shutdown complete.
I20260810 00:58:24.365480 28465 master_runner.cc:305] Clearing existing system tablet
I20260810 00:58:24.365991 28465 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:24.366176 28465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:24.372637 28465 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.001s
I20260810 00:58:24.379029 28681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:24.379727 28465 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:24.380281 28465 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.000s
I20260810 00:58:24.380522 28465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/wal
uuid: "6f8bda17b7544088b413ba1a962fdd6d"
format_stamp: "Formatted at 2026-08-10 00:58:24 on dist-test-slave-gp6n"
server_key: "81217c8951a6c35f2e3257356ec6f109"
server_key_iv: "76ffb5d12228f3b1a08c94ee336d3a62"
server_key_version: "encryptionkey@0"
I20260810 00:58:24.387470 28465 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:24.392527 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 27902
I20260810 00:58:24.400094 28465 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 00:58:24.400313 28465 log.cc:1196] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 00:58:24.400847 28465 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d: Deleting consensus metadata
I20260810 00:58:24.401209 28465 master_runner.cc:315] Copying system tablet from 127.20.81.190:33681
I20260810 00:58:24.419744 28465 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d: tablet copy: Beginning tablet copy session from remote peer at address 127.20.81.190:33681
I20260810 00:58:24.421638 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:33253
--local_ip_for_outbound_sockets=127.20.81.129
--tserver_master_addrs=127.20.81.190:33681
--webserver_port=38657
--webserver_interface=127.20.81.129
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.129.keytab
--principal=oryx/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:40295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.81.190:33681,127.20.81.189:46407
--tserver_master_addrs=127.20.81.190:33681,127.20.81.189:46407,127.20.81.188:45483 with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:58:24.441747 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:24.420995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46340 (local address 127.20.81.190:33681)
0810 00:58:24.421397 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:24.421409 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:24.422873 (+ 1464us) server_negotiation.cc:407] Connection header received
0810 00:58:24.423191 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:24.423202 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:24.423272 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:24.423396 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:24.425124 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.426370 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.429416 (+ 3046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.430559 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.430678 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:24.434367 (+ 3689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:24.434374 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:24.434378 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:24.434469 (+ 91us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:24.438232 (+ 3763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:24.439055 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:24.439065 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:24.439072 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:24.439342 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:24.439809 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:24.439815 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:24.439828 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:24.440208 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:24.440407 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:24.441008 (+ 601us) server_negotiation.cc:299] Negotiation successful
0810 00:58:24.441273 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211}
I20260810 00:58:24.442162 28687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:24.420951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.0.0.1:46340)
0810 00:58:24.422730 (+ 1779us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:24.422750 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:24.423000 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:24.423429 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:24.423434 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:24.424379 (+ 945us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:24.424916 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:24.424938 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.426608 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.426616 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:24.429218 (+ 2602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:24.429238 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.431028 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.431037 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:24.431426 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:24.432750 (+ 1324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:24.432771 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:24.434062 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:24.438397 (+ 4335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:24.438432 (+ 35us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:24.438439 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:24.438919 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:24.439485 (+ 566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:24.439491 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:24.439495 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:24.439682 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:24.440366 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:24.440371 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:24.440633 (+ 262us) client_negotiation.cc:769] Sending connection context
0810 00:58:24.440828 (+ 195us) client_negotiation.cc:240] Negotiation successful
0810 00:58:24.441249 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1571,"thread_start_us":1306,"threads_started":1}
I20260810 00:58:24.443127 27858 tablet_copy_service.cc:140] P 31ff0eea6672410291c03a7df56d1cf2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6f8bda17b7544088b413ba1a962fdd6d ({username='oryx', principal='oryx/127.20.81.188@KRBTEST.COM'} at 127.0.0.1:46340)
I20260810 00:58:24.443353 27858 tablet_copy_service.cc:161] P 31ff0eea6672410291c03a7df56d1cf2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6f8bda17b7544088b413ba1a962fdd6d at {username='oryx', principal='oryx/127.20.81.188@KRBTEST.COM'} at 127.0.0.1:46340: session id = 6f8bda17b7544088b413ba1a962fdd6d-00000000000000000000000000000000
I20260810 00:58:24.447166 27858 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 00:58:24.449106 28465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 00:58:24.456799 28465 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d: tablet copy: Starting download of 0 data blocks...
I20260810 00:58:24.457271 28465 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d: tablet copy: Starting download of 1 WAL segments...
I20260810 00:58:24.460229 28465 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 00:58:24.464797 27858 tablet_copy_service.cc:342] P 31ff0eea6672410291c03a7df56d1cf2: Request end of tablet copy session 6f8bda17b7544088b413ba1a962fdd6d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.20.81.188@KRBTEST.COM'} at 127.0.0.1:46340
I20260810 00:58:24.465051 27858 tablet_copy_service.cc:433] P 31ff0eea6672410291c03a7df56d1cf2: ending tablet copy session 6f8bda17b7544088b413ba1a962fdd6d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6f8bda17b7544088b413ba1a962fdd6d
I20260810 00:58:24.477602 28465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:24.480867 28689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:24.483861 28691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:24.491789 28465 server_base.cc:1034] running on GCE node
W20260810 00:58:24.492602 28695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:24.493279 28465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:24.494349 28465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:24.495640 28465 hybrid_clock.cc:648] HybridClock initialized: now 1786323504495567 us; error 48 us; skew 500 ppm
I20260810 00:58:24.495968 28465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:24 dist-test-slave-gp6n krb5kdc[27789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323504, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:24.499239 28465 init.cc:377] Logged in from keytab as oryx/127.20.81.188@KRBTEST.COM (short username oryx)
I20260810 00:58:24.504262 28465 webserver.cc:460] Webserver started at http://127.20.81.188:41195/ using document root <none> and password file <none>
I20260810 00:58:24.504781 28465 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:24.505013 28465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:24.510748 28465 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260810 00:58:24.514927 28704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:24.516003 28465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 00:58:24.516258 28465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/wal
uuid: "6f8bda17b7544088b413ba1a962fdd6d"
format_stamp: "Formatted at 2026-08-10 00:58:24 on dist-test-slave-gp6n"
server_key: "81217c8951a6c35f2e3257356ec6f109"
server_key_iv: "76ffb5d12228f3b1a08c94ee336d3a62"
server_key_version: "encryptionkey@0"
I20260810 00:58:24.516510 28465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:24.553058 28465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:24.557524 28465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:24.594770 28465 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.188:45483
I20260810 00:58:24.594853 28759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.188:45483 every 8 connection(s)
I20260810 00:58:24.595878 28465 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/master-2/data/info.pb
I20260810 00:58:24.603497 28760 sys_catalog.cc:263] Verifying existing consensus state
I20260810 00:58:24.607146 28760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d: Bootstrap starting.
I20260810 00:58:24.625962 28760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 00:58:24.626637 28760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d: Bootstrap complete.
I20260810 00:58:24.627754 28760 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: NON_VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: true } }
I20260810 00:58:24.628072 28760 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6f8bda17b7544088b413ba1a962fdd6d, State: Initialized, Role: LEARNER
I20260810 00:58:24.628337 28760 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: NON_VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: true } }
I20260810 00:58:24.629607 28760 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:58:24.629361 28763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: NON_VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: true } } }
I20260810 00:58:24.629981 28763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [sys.catalog]: This master's current role is: LEARNER
I20260810 00:58:24.638197 28777 catalog_manager.cc:1259] Loaded cluster ID: 8c10a06edd3847a3bd451ed9039cadf6
I20260810 00:58:24.638307 28777 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d: loading cluster ID for follower catalog manager: success
I20260810 00:58:24.642191 28777 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d: acquiring CA information for follower catalog manager: success
I20260810 00:58:24.644847 28777 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 00:58:24.860355 28688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:24.860846 28688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:24.861111 28688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:24.876240 28688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:24.876441 28688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:24.876659 28688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 00:58:24.889084 28688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:40295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.129.keytab
--principal=oryx/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:33253
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=38657
--tserver_master_addrs=127.20.81.190:33681,127.20.81.189:46407,127.20.81.188:45483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:24.890677 28688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:24.892164 28688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:24.910346 28783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:24.910521 28784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:24.912062 28786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:24.913117 28688 server_base.cc:1034] running on GCE node
I20260810 00:58:24.914147 28688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:24.915645 28688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:24.916424 28787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:24.906764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:45483 (local address 127.0.0.1:48564)
0810 00:58:24.908687 (+ 1923us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:24.908709 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:24.908892 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:24.909489 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:24.909493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:24.909517 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:24.909935 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:24.909951 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.912582 (+ 2631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.912588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:24.914183 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:24.914197 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.915759 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.915768 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:24.915955 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:24.915997 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 00:58:24.916079 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 00:58:24.916181 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1801,"thread_start_us":1668,"threads_started":1}
I20260810 00:58:24.916704 28789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:24.906826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48564 (local address 127.20.81.188:45483)
0810 00:58:24.908374 (+ 1548us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:24.908383 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:24.908985 (+ 602us) server_negotiation.cc:407] Connection header received
0810 00:58:24.909101 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:24.909108 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:24.909219 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:24.909438 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:24.910071 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.912440 (+ 2369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.914348 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:24.915532 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:24.915689 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:24.915911 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:24.916128 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 00:58:24.916287 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1352,"thread_start_us":1135,"threads_started":1}
I20260810 00:58:24.916904 28688 hybrid_clock.cc:648] HybridClock initialized: now 1786323504916829 us; error 51 us; skew 500 ppm
I20260810 00:58:24.917237 28688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:58:24.917795 28734 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260810 00:58:24.918556 28763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31ff0eea6672410291c03a7df56d1cf2. Latest consensus state: current_term: 1 leader_uuid: "31ff0eea6672410291c03a7df56d1cf2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: NON_VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: true } } }
I20260810 00:58:24.918923 28763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [sys.catalog]: This master's current role is: LEARNER
Aug 10 00:58:24 dist-test-slave-gp6n krb5kdc[27789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323504, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:24.922091 28688 init.cc:377] Logged in from keytab as oryx/127.20.81.129@KRBTEST.COM (short username oryx)
I20260810 00:58:24.924126 28688 webserver.cc:460] Webserver started at http://127.20.81.129:38657/ using document root <none> and password file <none>
I20260810 00:58:24.925101 28688 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:24.925279 28688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:24.932472 28688 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 00:58:24.936344 28796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:24.937857 28688 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 00:58:24.938053 28688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "64786d2c70b344f8954cb3344937e3b7"
format_stamp: "Formatted at 2026-08-10 00:58:21 on dist-test-slave-gp6n"
server_key: "920e036602a9da6ecf45fc0d8d077a05"
server_key_iv: "1bd426dd64a0b693fc82b75a5dfea0e5"
server_key_version: "encryptionkey@0"
I20260810 00:58:24.939630 28688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:24.964083 28688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:24.969993 28688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:24.970494 28688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:24.972046 28688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:24.973883 28688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:24.974080 28688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:24.974231 28688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:24.974344 28688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:25.050668 28688 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:33253
I20260810 00:58:25.050904 28920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:33253 every 8 connection(s)
I20260810 00:58:25.052464 28688 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 00:58:25.058936 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 28688
I20260810 00:58:25.059504 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 28053
Aug 10 00:58:25 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323504, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.189@KRBTEST.COM
Aug 10 00:58:25 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323504, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.188@KRBTEST.COM
I20260810 00:58:25.101689 28926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:25.056955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:54839 (local address 127.20.81.189:46407)
0810 00:58:25.058091 (+ 1136us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:25.058098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:25.070496 (+ 12398us) server_negotiation.cc:407] Connection header received
0810 00:58:25.071302 (+ 806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:25.071309 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:25.071419 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:25.071555 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:25.074091 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.075998 (+ 1907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.079934 (+ 3936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.081041 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.081186 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:25.089482 (+ 8296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:25.089493 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:25.089502 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:25.089561 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:25.095960 (+ 6399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:25.097153 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:25.097165 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:25.097171 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:25.097490 (+ 319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:25.098230 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:25.098235 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:25.098238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:25.098759 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:25.098932 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:25.100805 (+ 1873us) server_negotiation.cc:299] Negotiation successful
0810 00:58:25.101057 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":971,"thread_start_us":744,"threads_started":1}
I20260810 00:58:25.104329 28928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:25.057832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:46407 (local address 127.20.81.129:54839)
0810 00:58:25.067864 (+ 10032us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:25.067940 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:25.071008 (+ 3068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:25.071890 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:25.071903 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:25.072685 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:25.073800 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:25.073821 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.076355 (+ 2534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.076372 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:25.079705 (+ 3333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:25.079728 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.081703 (+ 1975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.081712 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:25.082166 (+ 454us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:25.083441 (+ 1275us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:25.083484 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:25.089072 (+ 5588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:25.096161 (+ 7089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:25.096174 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:25.096195 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:25.096979 (+ 784us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:25.097662 (+ 683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:25.097668 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:25.097672 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:25.098032 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:25.099029 (+ 997us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:25.099038 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:25.099635 (+ 597us) client_negotiation.cc:769] Sending connection context
0810 00:58:25.100010 (+ 375us) client_negotiation.cc:240] Negotiation successful
0810 00:58:25.102231 (+ 2221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9540,"thread_start_us":9181,"threads_started":1}
Aug 10 00:58:25 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323504, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM
I20260810 00:58:25.105582 28789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:25.056927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:44863 (local address 127.20.81.188:45483)
0810 00:58:25.057141 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:25.057147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:25.074552 (+ 17405us) server_negotiation.cc:407] Connection header received
0810 00:58:25.074884 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:25.074893 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:25.075000 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:25.075077 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:25.077515 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.079341 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.084874 (+ 5533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.086038 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.086162 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:25.094082 (+ 7920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:25.094091 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:25.094098 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:25.094148 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:25.099598 (+ 5450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:25.101709 (+ 2111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:25.101718 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:25.101723 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:25.102228 (+ 505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:25.102825 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:25.102832 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:25.102835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:25.103320 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:25.103448 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:25.104208 (+ 760us) server_negotiation.cc:299] Negotiation successful
0810 00:58:25.104961 (+ 753us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
Aug 10 00:58:25 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.81.130@KRBTEST.COM
I20260810 00:58:25.107043 28929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:25.059360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:45483 (local address 127.20.81.129:44863)
0810 00:58:25.074382 (+ 15022us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:25.074466 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:25.074670 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:25.075373 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:25.075381 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:25.076638 (+ 1257us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:25.077312 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:25.077328 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.079628 (+ 2300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.079636 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:25.084671 (+ 5035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:25.084690 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.087652 (+ 2962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.087661 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:25.087883 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:25.089576 (+ 1693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:25.089603 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:25.093735 (+ 4132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:25.099818 (+ 6083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:25.099824 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:25.099827 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:25.101538 (+ 1711us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:25.102385 (+ 847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:25.102389 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:25.102392 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:25.102675 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:25.103547 (+ 872us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:25.103552 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:25.103830 (+ 278us) client_negotiation.cc:769] Sending connection context
0810 00:58:25.105888 (+ 2058us) client_negotiation.cc:240] Negotiation successful
0810 00:58:25.106318 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14871,"spinlock_wait_cycles":24320,"thread_start_us":14603,"threads_started":1}
I20260810 00:58:25.109357 28921 heartbeater.cc:347] Connected to a master server at 127.20.81.188:45483
I20260810 00:58:25.109687 28921 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:25.110560 28924 heartbeater.cc:347] Connected to a master server at 127.20.81.189:46407
I20260810 00:58:25.110821 28924 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:25.111342 28921 heartbeater.cc:511] Master 127.20.81.188:45483 requested a full tablet report, sending...
I20260810 00:58:25.111948 28924 heartbeater.cc:511] Master 127.20.81.189:46407 requested a full tablet report, sending...
I20260810 00:58:25.115056 28509 ts_manager.cc:194] Registered new tserver with Master: 64786d2c70b344f8954cb3344937e3b7 (127.20.81.129:33253)
I20260810 00:58:25.115340 28722 ts_manager.cc:194] Registered new tserver with Master: 64786d2c70b344f8954cb3344937e3b7 (127.20.81.129:33253)
I20260810 00:58:25.115644 28932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:25.074594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.129:51969)
0810 00:58:25.088472 (+ 13878us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:25.088493 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:25.088648 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:25.091802 (+ 3154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:25.091814 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:25.092842 (+ 1028us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:25.093663 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:25.093677 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.095513 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.095519 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:25.097626 (+ 2107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:25.097641 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.099581 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.099590 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:25.099916 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:25.102239 (+ 2323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:25.102265 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:25.106080 (+ 3815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:25.111252 (+ 5172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:25.111257 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:25.111275 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:25.112058 (+ 783us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:25.112779 (+ 721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:25.112784 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:25.112788 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:25.113039 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:25.114093 (+ 1054us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:25.114100 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:25.114339 (+ 239us) client_negotiation.cc:769] Sending connection context
0810 00:58:25.114523 (+ 184us) client_negotiation.cc:240] Negotiation successful
0810 00:58:25.114857 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13694,"thread_start_us":12980,"threads_started":1}
I20260810 00:58:25.116849 28933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:25.074750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:51969 (local address 127.20.81.190:33681)
0810 00:58:25.091435 (+ 16685us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:25.091443 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:25.091465 (+ 22us) server_negotiation.cc:407] Connection header received
0810 00:58:25.091543 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:25.091549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:25.091633 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:25.091756 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:25.093850 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.095314 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.097837 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.098921 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.099063 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:25.106330 (+ 7267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:25.106335 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:25.106339 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:25.106372 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:25.111061 (+ 4689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:25.112177 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:25.112182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:25.112186 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:25.112358 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:25.113209 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:25.113214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:25.113217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:25.113725 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:25.113961 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:25.115930 (+ 1969us) server_negotiation.cc:299] Negotiation successful
0810 00:58:25.116167 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15962,"spinlock_wait_cycles":278912,"thread_start_us":16281,"threads_started":1}
I20260810 00:58:25.117929 28923 heartbeater.cc:347] Connected to a master server at 127.20.81.190:33681
I20260810 00:58:25.118098 28923 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:25.118101 28931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:25.060653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40686 (local address 127.20.81.130:36539)
0810 00:58:25.087084 (+ 26431us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:25.087092 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:25.087117 (+ 25us) server_negotiation.cc:407] Connection header received
0810 00:58:25.087171 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:25.087177 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:25.087421 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:25.091662 (+ 4241us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:25.092087 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.094068 (+ 1981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.096762 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.097922 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.104915 (+ 6993us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:25.108467 (+ 3552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:25.108487 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:25.108511 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:25.108576 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:25.113235 (+ 4659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:25.114486 (+ 1251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:25.114495 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:25.114498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:25.114729 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:25.115308 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:25.115315 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:25.115319 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:25.115785 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:25.116154 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:25.116956 (+ 802us) server_negotiation.cc:299] Negotiation successful
0810 00:58:25.117389 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26262,"thread_start_us":27068,"threads_started":1}
I20260810 00:58:25.118927 28923 heartbeater.cc:511] Master 127.20.81.190:33681 requested a full tablet report, sending...
I20260810 00:58:25.119763 28193 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:25.120187 27838 ts_manager.cc:194] Re-registered known tserver with Master: 64786d2c70b344f8954cb3344937e3b7 (127.20.81.129:33253)
I20260810 00:58:25.121467 27838 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:51969
I20260810 00:58:25.224968 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 28053
I20260810 00:58:25.247272 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:36539
--local_ip_for_outbound_sockets=127.20.81.130
--tserver_master_addrs=127.20.81.190:33681
--webserver_port=40595
--webserver_interface=127.20.81.130
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.130.keytab
--principal=oryx/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:40295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.81.190:33681,127.20.81.189:46407
--tserver_master_addrs=127.20.81.190:33681,127.20.81.189:46407,127.20.81.188:45483 with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:58:25.495512 28671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31ff0eea6672410291c03a7df56d1cf2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: NON_VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: true } } }
I20260810 00:58:25.496007 28671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:25.497664 28936 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2: attempting to promote NON_VOTER 6f8bda17b7544088b413ba1a962fdd6d to VOTER
I20260810 00:58:25.498248 28936 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: false } }
I20260810 00:58:25.500679 28519 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 1 FOLLOWER]: Refusing update from remote peer 31ff0eea6672410291c03a7df56d1cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 00:58:25.500699 28734 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 1 LEARNER]: Refusing update from remote peer 31ff0eea6672410291c03a7df56d1cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 00:58:25.501296 28936 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 00:58:25.501719 28937 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 00:58:25.505925 28671 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 6f8bda17b7544088b413ba1a962fdd6d (127.20.81.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: false } } }
I20260810 00:58:25.506690 28734 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 6f8bda17b7544088b413ba1a962fdd6d (127.20.81.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: false } } }
I20260810 00:58:25.507030 28519 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 6f8bda17b7544088b413ba1a962fdd6d (127.20.81.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: false } } }
I20260810 00:58:25.510524 28671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "31ff0eea6672410291c03a7df56d1cf2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: false } } }
I20260810 00:58:25.510820 28671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:25.511206 28671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31ff0eea6672410291c03a7df56d1cf2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: false } } }
I20260810 00:58:25.511502 28671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:25.511703 28936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31ff0eea6672410291c03a7df56d1cf2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: false } } }
I20260810 00:58:25.511963 28936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:25.515448 28945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31ff0eea6672410291c03a7df56d1cf2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: false } } }
I20260810 00:58:25.515828 28945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:58:25.516433 28944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31ff0eea6672410291c03a7df56d1cf2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: false } } }
I20260810 00:58:25.516880 28944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [sys.catalog]: This master's current role is: FOLLOWER
W20260810 00:58:25.614888 28935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:25.615159 28935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:25.615253 28935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:25.622891 28935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:25.622996 28935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:25.623107 28935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 00:58:25.633229 28935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:40295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.130.keytab
--principal=oryx/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:36539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=40595
--tserver_master_addrs=127.20.81.190:33681,127.20.81.189:46407,127.20.81.188:45483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:25.634563 28935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:25.635862 28935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:25.650897 28952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:25.650962 28953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:25.651235 28956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:25.651897 28935 server_base.cc:1034] running on GCE node
I20260810 00:58:25.652582 28935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:25.653527 28935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:25.654804 28935 hybrid_clock.cc:648] HybridClock initialized: now 1786323505654762 us; error 36 us; skew 500 ppm
I20260810 00:58:25.655128 28935 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:25 dist-test-slave-gp6n krb5kdc[27789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323505, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:25.658576 28935 init.cc:377] Logged in from keytab as oryx/127.20.81.130@KRBTEST.COM (short username oryx)
I20260810 00:58:25.659996 28935 webserver.cc:460] Webserver started at http://127.20.81.130:40595/ using document root <none> and password file <none>
I20260810 00:58:25.660710 28935 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:25.660873 28935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:25.666599 28935 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 00:58:25.669229 28963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:25.670681 28935 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:58:25.670866 28935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "4b439751b0f64ffd8322d2894db470ac"
format_stamp: "Formatted at 2026-08-10 00:58:22 on dist-test-slave-gp6n"
server_key: "6d1cc4e186c2a68ad0188278a40d1810"
server_key_iv: "223b3701ff5b52c3f975465e49763574"
server_key_version: "encryptionkey@0"
I20260810 00:58:25.672433 28935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:25.692979 28935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:25.696931 28935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:25.697211 28935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:25.698192 28935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:25.699594 28935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:25.699716 28935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:25.699851 28935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:25.699934 28935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:25.734985 28935 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:36539
I20260810 00:58:25.735059 29093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:36539 every 8 connection(s)
I20260810 00:58:25.736443 28935 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 00:58:25.744083 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 28935
I20260810 00:58:25.744424 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 28205
Aug 10 00:58:25 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323505, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.189@KRBTEST.COM
Aug 10 00:58:25 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323505, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.188@KRBTEST.COM
Aug 10 00:58:25 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.81.131@KRBTEST.COM
Aug 10 00:58:25 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323505, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM
I20260810 00:58:25.771544 29098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:25.739312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:45221 (local address 127.20.81.189:46407)
0810 00:58:25.739731 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:25.739736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:25.745311 (+ 5575us) server_negotiation.cc:407] Connection header received
0810 00:58:25.745486 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:25.745492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:25.745611 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:25.745730 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:25.748823 (+ 3093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.750386 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.753983 (+ 3597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.754913 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.755093 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:25.762809 (+ 7716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:25.762814 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:25.762817 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:25.762892 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:25.767108 (+ 4216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:25.768262 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:25.768268 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:25.768272 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:25.768423 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:25.769015 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:25.769020 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:25.769024 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:25.769419 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:25.769560 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:25.770800 (+ 1240us) server_negotiation.cc:299] Negotiation successful
0810 00:58:25.771071 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":193,"threads_started":1}
I20260810 00:58:25.773506 29102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:25.740003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:46407 (local address 127.20.81.130:45221)
0810 00:58:25.745193 (+ 5190us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:25.745217 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:25.745358 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:25.746176 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:25.746192 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:25.747403 (+ 1211us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:25.748547 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:25.748585 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.750660 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.750668 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:25.753744 (+ 3076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:25.753765 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.755535 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.755545 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:25.755944 (+ 399us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:25.757002 (+ 1058us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:25.757045 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:25.762534 (+ 5489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:25.767292 (+ 4758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:25.767302 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:25.767319 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:25.768107 (+ 788us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:25.768580 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:25.768587 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:25.768591 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:25.768874 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:25.769733 (+ 859us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:25.769742 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:25.770243 (+ 501us) client_negotiation.cc:769] Sending connection context
0810 00:58:25.771479 (+ 1236us) client_negotiation.cc:240] Negotiation successful
0810 00:58:25.772011 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5050,"mutex_wait_us":57,"thread_start_us":19680,"threads_started":1}
I20260810 00:58:25.774258 29099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:25.740003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:45483 (local address 127.20.81.130:45111)
0810 00:58:25.740957 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:25.741013 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:25.743977 (+ 2964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:25.756659 (+ 12682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:25.756667 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:25.757230 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:25.757564 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:25.757577 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.759327 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.759333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:25.761019 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:25.761033 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.762065 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.762071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:25.762239 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:25.763106 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:25.763127 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:25.766165 (+ 3038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:25.770104 (+ 3939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:25.770110 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:25.770113 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:25.770670 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:25.771113 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:25.771119 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:25.771124 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:25.771277 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:25.771897 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:25.771902 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:25.772069 (+ 167us) client_negotiation.cc:769] Sending connection context
0810 00:58:25.772182 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 00:58:25.772442 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"mutex_wait_us":1,"thread_start_us":404,"threads_started":1}
I20260810 00:58:25.774724 29101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:25.740301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:45111 (local address 127.20.81.188:45483)
0810 00:58:25.756279 (+ 15978us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:25.756287 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:25.756309 (+ 22us) server_negotiation.cc:407] Connection header received
0810 00:58:25.756374 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:25.756378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:25.756464 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:25.757738 (+ 1274us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:25.757948 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.759162 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.761218 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.761842 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.762859 (+ 1017us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:25.766382 (+ 3523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:25.766391 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:25.766424 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:25.766464 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:25.769930 (+ 3466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:25.770824 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:25.770831 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:25.770836 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:25.770969 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:25.771433 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:25.771441 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:25.771447 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:25.771758 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:25.773510 (+ 1752us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:25.773666 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 00:58:25.773890 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14463,"thread_start_us":15789,"threads_started":1}
I20260810 00:58:25.775978 29094 heartbeater.cc:347] Connected to a master server at 127.20.81.188:45483
I20260810 00:58:25.776403 29094 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:25.775969 29096 heartbeater.cc:347] Connected to a master server at 127.20.81.189:46407
I20260810 00:58:25.776842 29096 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:25.777627 29094 heartbeater.cc:511] Master 127.20.81.188:45483 requested a full tablet report, sending...
I20260810 00:58:25.777659 29096 heartbeater.cc:511] Master 127.20.81.189:46407 requested a full tablet report, sending...
I20260810 00:58:25.778668 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:25.745591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49720 (local address 127.20.81.131:35525)
0810 00:58:25.755387 (+ 9796us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:25.755394 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:25.755420 (+ 26us) server_negotiation.cc:407] Connection header received
0810 00:58:25.755476 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:25.755483 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:25.755666 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:25.755845 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:25.758073 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.759623 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.761549 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.762082 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.762224 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:25.768625 (+ 6401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:25.768639 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:25.768657 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:25.768699 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:25.774205 (+ 5506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:25.775188 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:25.775197 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:25.775200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:25.775370 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:25.775821 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:25.775828 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:25.775831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:25.776255 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:25.776596 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:25.777938 (+ 1342us) server_negotiation.cc:299] Negotiation successful
0810 00:58:25.778210 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9629,"thread_start_us":9475,"threads_started":1}
I20260810 00:58:25.779574 28722 ts_manager.cc:194] Registered new tserver with Master: 4b439751b0f64ffd8322d2894db470ac (127.20.81.130:36539)
I20260810 00:58:25.779578 28509 ts_manager.cc:194] Registered new tserver with Master: 4b439751b0f64ffd8322d2894db470ac (127.20.81.130:36539)
I20260810 00:58:25.780030 29103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:25.741861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:39935 (local address 127.20.81.190:33681)
0810 00:58:25.744742 (+ 2881us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:25.744751 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:25.759851 (+ 15100us) server_negotiation.cc:407] Connection header received
0810 00:58:25.760032 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:25.760038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:25.760119 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:25.760232 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:25.761536 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.762653 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.765642 (+ 2989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.766331 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.766449 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:25.770692 (+ 4243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:25.770701 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:25.770707 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:25.770762 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:25.774205 (+ 3443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:25.776653 (+ 2448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:25.776662 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:25.776669 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:25.776835 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:25.777980 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:25.777986 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:25.777989 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:25.778338 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:25.778551 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:25.779323 (+ 772us) server_negotiation.cc:299] Negotiation successful
0810 00:58:25.779577 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2712,"thread_start_us":2646,"threads_started":1}
I20260810 00:58:25.780042 28343 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:25.780961 29100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:25.741510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.130:39935)
0810 00:58:25.759747 (+ 18237us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:25.759767 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:25.759904 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:25.760275 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:25.760283 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:25.760920 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:25.761387 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:25.761404 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.763112 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.763117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:25.765462 (+ 2345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:25.765476 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:25.766763 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:25.766769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:25.766954 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:25.767513 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:25.767546 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:25.770476 (+ 2930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:25.775940 (+ 5464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:25.775947 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:25.775952 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:25.776536 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:25.777534 (+ 998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:25.777539 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:25.777543 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:25.777820 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:25.778612 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:25.778617 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:25.779015 (+ 398us) client_negotiation.cc:769] Sending connection context
0810 00:58:25.780402 (+ 1387us) client_negotiation.cc:240] Negotiation successful
0810 00:58:25.780697 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18082,"mutex_wait_us":63,"thread_start_us":3439,"threads_started":1}
I20260810 00:58:25.781780 29095 heartbeater.cc:347] Connected to a master server at 127.20.81.190:33681
I20260810 00:58:25.781899 29095 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:25.782364 29095 heartbeater.cc:511] Master 127.20.81.190:33681 requested a full tablet report, sending...
I20260810 00:58:25.783407 27838 ts_manager.cc:194] Re-registered known tserver with Master: 4b439751b0f64ffd8322d2894db470ac (127.20.81.130:36539)
I20260810 00:58:25.784646 27838 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:39935
I20260810 00:58:25.869957 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 28205
I20260810 00:58:25.890302 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:35525
--local_ip_for_outbound_sockets=127.20.81.131
--tserver_master_addrs=127.20.81.190:33681
--webserver_port=41941
--webserver_interface=127.20.81.131
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.131.keytab
--principal=oryx/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:40295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.81.190:33681,127.20.81.189:46407
--tserver_master_addrs=127.20.81.190:33681,127.20.81.189:46407,127.20.81.188:45483 with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:58:26.124432 28923 heartbeater.cc:503] Master 127.20.81.190:33681 was elected leader, sending a full tablet report...
W20260810 00:58:26.334262 29107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:26.334625 29107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:26.334726 29107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:26.341818 29107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:26.341915 29107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:26.342023 29107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 00:58:26.351890 29107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:40295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.131.keytab
--principal=oryx/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:35525
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=41941
--tserver_master_addrs=127.20.81.190:33681,127.20.81.189:46407,127.20.81.188:45483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:26.353287 29107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:26.354669 29107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:26.368180 29112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:26.368094 29115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:26.368542 29113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:26.369664 29107 server_base.cc:1034] running on GCE node
I20260810 00:58:26.370296 29107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:26.371271 29107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:26.372478 29107 hybrid_clock.cc:648] HybridClock initialized: now 1786323506372416 us; error 51 us; skew 500 ppm
I20260810 00:58:26.372812 29107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:26 dist-test-slave-gp6n krb5kdc[27789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323506, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:26.376384 29107 init.cc:377] Logged in from keytab as oryx/127.20.81.131@KRBTEST.COM (short username oryx)
I20260810 00:58:26.377992 29107 webserver.cc:460] Webserver started at http://127.20.81.131:41941/ using document root <none> and password file <none>
I20260810 00:58:26.378746 29107 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:26.378870 29107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:26.384279 29107 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:58:26.387099 29122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:26.388442 29107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:58:26.388622 29107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "94f458e21af3447e8489ca2b0f0c2e07"
format_stamp: "Formatted at 2026-08-10 00:58:22 on dist-test-slave-gp6n"
server_key: "4f8ca886f049127d7331382408bcecf5"
server_key_iv: "9f86909ae809e03cb2a6fcdcc11a3d93"
server_key_version: "encryptionkey@0"
I20260810 00:58:26.389654 29107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:26.419992 29107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:26.424297 29107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:26.424609 29107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:26.425618 29107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:26.427094 29107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:26.427182 29107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:26.427341 29107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:26.427448 29107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:26.462499 29107 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:35525
I20260810 00:58:26.462589 29247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:35525 every 8 connection(s)
I20260810 00:58:26.463979 29107 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 00:58:26.469538 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 29107
Aug 10 00:58:26 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323506, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM
Aug 10 00:58:26 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323506, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.188@KRBTEST.COM
Aug 10 00:58:26 dist-test-slave-gp6n krb5kdc[27789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323506, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.189@KRBTEST.COM
I20260810 00:58:26.495168 29252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:26.466924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:60967 (local address 127.20.81.190:33681)
0810 00:58:26.467591 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:26.467601 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:26.468879 (+ 1278us) server_negotiation.cc:407] Connection header received
0810 00:58:26.472599 (+ 3720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:26.472608 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:26.472725 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:26.472831 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:26.475940 (+ 3109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.477467 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.480363 (+ 2896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.480994 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.481095 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:26.486022 (+ 4927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:26.486032 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:26.486040 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:26.486087 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:26.490048 (+ 3961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:26.491876 (+ 1828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:26.491884 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:26.491889 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:26.492050 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:26.492695 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:26.492702 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:26.492707 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:26.493090 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:26.493240 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:26.494511 (+ 1271us) server_negotiation.cc:299] Negotiation successful
0810 00:58:26.494759 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":276,"threads_started":1}
I20260810 00:58:26.498867 29254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:26.469135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:45483 (local address 127.20.81.131:46151)
0810 00:58:26.478101 (+ 8966us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:26.478124 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:26.478244 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:26.479035 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:26.479040 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:26.479665 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:26.480033 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:26.480046 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.481960 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.481965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:26.483686 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:26.483701 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.485600 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.485609 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:26.485769 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:26.486383 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:26.486428 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:26.489301 (+ 2873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:26.493652 (+ 4351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:26.493657 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:26.493660 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:26.494246 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:26.495787 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:26.495792 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:26.495795 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:26.495957 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:26.496748 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:26.496756 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:26.496987 (+ 231us) client_negotiation.cc:769] Sending connection context
0810 00:58:26.497153 (+ 166us) client_negotiation.cc:240] Negotiation successful
0810 00:58:26.497520 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8820,"mutex_wait_us":1,"thread_start_us":8461,"threads_started":1}
I20260810 00:58:26.501164 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:26.471768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:46151 (local address 127.20.81.188:45483)
0810 00:58:26.478669 (+ 6901us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:26.478675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:26.478694 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:58:26.478752 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:26.478758 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:26.478859 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:26.480488 (+ 1629us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:26.480647 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.481761 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.483909 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.485228 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.485354 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:26.489531 (+ 4177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:26.489538 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:26.489543 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:26.489588 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:26.493313 (+ 3725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:26.495455 (+ 2142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:26.495461 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:26.495464 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:26.495628 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:26.496103 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:26.496109 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:26.496114 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:26.496507 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:26.496656 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:26.500202 (+ 3546us) server_negotiation.cc:299] Negotiation successful
0810 00:58:26.500516 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6778,"thread_start_us":6671,"threads_started":1}
I20260810 00:58:26.502820 29248 heartbeater.cc:347] Connected to a master server at 127.20.81.188:45483
I20260810 00:58:26.503225 29248 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:26.503533 29255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:26.467621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.131:60967)
0810 00:58:26.468711 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:26.468769 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:26.472169 (+ 3400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:26.473616 (+ 1447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:26.473632 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:26.474650 (+ 1018us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:26.475675 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:26.475708 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.477730 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.477735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:26.480172 (+ 2437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:26.480188 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.481460 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.481466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:26.481719 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:26.482598 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:26.482643 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:26.485737 (+ 3094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:26.490238 (+ 4501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:26.490545 (+ 307us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:26.490566 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:26.491717 (+ 1151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:26.492235 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:26.492242 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:26.492246 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:26.492540 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:26.493363 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:26.493374 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:26.493893 (+ 519us) client_negotiation.cc:769] Sending connection context
0810 00:58:26.494353 (+ 460us) client_negotiation.cc:240] Negotiation successful
0810 00:58:26.494995 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"mutex_wait_us":41,"thread_start_us":420,"threads_started":1}
I20260810 00:58:26.503898 29253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:26.466920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:43149 (local address 127.20.81.189:46407)
0810 00:58:26.483255 (+ 16335us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:26.483264 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:26.483290 (+ 26us) server_negotiation.cc:407] Connection header received
0810 00:58:26.483368 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:26.483376 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:26.483506 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:26.485230 (+ 1724us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:26.485449 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.486726 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.489392 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.490291 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.491242 (+ 951us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:26.495302 (+ 4060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:26.495310 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:26.495314 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:26.495356 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:26.499589 (+ 4233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:26.500532 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:26.500541 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:26.500547 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:26.500720 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:26.501241 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:26.501248 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:26.501254 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:26.501611 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:26.502925 (+ 1314us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:26.503073 (+ 148us) server_negotiation.cc:299] Negotiation successful
0810 00:58:26.503347 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16166,"thread_start_us":16000,"threads_started":1}
I20260810 00:58:26.504254 29248 heartbeater.cc:511] Master 127.20.81.188:45483 requested a full tablet report, sending...
I20260810 00:58:26.505055 29249 heartbeater.cc:347] Connected to a master server at 127.20.81.190:33681
I20260810 00:58:26.505187 29249 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:26.505407 29256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:26.467953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:46407 (local address 127.20.81.131:43149)
0810 00:58:26.477927 (+ 9974us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:26.477948 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:26.478093 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:26.483723 (+ 5630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:26.483736 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:26.484621 (+ 885us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:26.485038 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:26.485051 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.486940 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.486948 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:26.489135 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:26.489154 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.490571 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.490580 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:26.490835 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:26.491565 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:26.491588 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:26.495081 (+ 3493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:26.499757 (+ 4676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:26.499766 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:26.499772 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:26.500345 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:26.500865 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:26.500873 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:26.500879 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:26.501068 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:26.501761 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:26.501769 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:26.501952 (+ 183us) client_negotiation.cc:769] Sending connection context
0810 00:58:26.502106 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 00:58:26.502498 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9817,"thread_start_us":354,"threads_started":1}
I20260810 00:58:26.505785 29249 heartbeater.cc:511] Master 127.20.81.190:33681 requested a full tablet report, sending...
I20260810 00:58:26.506589 29250 heartbeater.cc:347] Connected to a master server at 127.20.81.189:46407
I20260810 00:58:26.506759 29250 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:26.507293 29250 heartbeater.cc:511] Master 127.20.81.189:46407 requested a full tablet report, sending...
I20260810 00:58:26.507519 27838 ts_manager.cc:194] Re-registered known tserver with Master: 94f458e21af3447e8489ca2b0f0c2e07 (127.20.81.131:35525)
I20260810 00:58:26.507642 28722 ts_manager.cc:194] Registered new tserver with Master: 94f458e21af3447e8489ca2b0f0c2e07 (127.20.81.131:35525)
I20260810 00:58:26.508464 27838 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:60967
I20260810 00:58:26.508704 28509 ts_manager.cc:194] Registered new tserver with Master: 94f458e21af3447e8489ca2b0f0c2e07 (127.20.81.131:35525)
I20260810 00:58:26.531342 29253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:26.495987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41492 (local address 127.20.81.189:46407)
0810 00:58:26.504280 (+ 8293us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:26.504287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:26.515333 (+ 11046us) server_negotiation.cc:407] Connection header received
0810 00:58:26.515503 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:26.515512 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:26.515607 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:26.515713 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:26.518389 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.519616 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.521224 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.521636 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.521767 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:26.525626 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:26.525632 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:26.525637 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:26.525677 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:26.528373 (+ 2696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:26.529041 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:26.529048 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:26.529052 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:26.529175 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:26.529609 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:26.529616 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:26.529620 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:26.530007 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:26.530260 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:26.530727 (+ 467us) server_negotiation.cc:299] Negotiation successful
0810 00:58:26.530960 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8145,"thread_start_us":18278,"threads_started":1}
I20260810 00:58:26.535032 29252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:26.496548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46348 (local address 127.20.81.190:33681)
0810 00:58:26.499788 (+ 3240us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:26.499796 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:26.516048 (+ 16252us) server_negotiation.cc:407] Connection header received
0810 00:58:26.516299 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:26.516306 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:26.516389 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:26.516463 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:26.519018 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.520153 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.522355 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.522896 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.523044 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:26.525657 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:26.525663 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:26.525668 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:26.525707 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:26.529213 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:26.530471 (+ 1258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:26.530477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:26.530480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:26.530653 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:26.533301 (+ 2648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:26.533309 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:26.533314 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:26.533739 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:26.533951 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:26.534333 (+ 382us) server_negotiation.cc:299] Negotiation successful
0810 00:58:26.534691 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3079,"thread_start_us":18054,"threads_started":1}
I20260810 00:58:26.535948 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:26.516125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48580 (local address 127.20.81.188:45483)
0810 00:58:26.516333 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:26.516342 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:26.519518 (+ 3176us) server_negotiation.cc:407] Connection header received
0810 00:58:26.519647 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:26.519655 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:26.519744 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:26.519833 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:26.522277 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.523439 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.525609 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.526273 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.526360 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:26.529790 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:26.529796 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:26.529801 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:26.529840 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:26.533059 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:26.533790 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:26.533813 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:26.533819 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:26.533970 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:26.534457 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:26.534461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:26.534465 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:26.534766 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:26.534936 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:26.535422 (+ 486us) server_negotiation.cc:299] Negotiation successful
0810 00:58:26.535648 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 00:58:26.539179 27838 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46348:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:58:26.546152 27838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:58:26.591645 29280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:26.565373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:36539 (local address 127.0.0.1:40694)
0810 00:58:26.565981 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:26.566006 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:26.566132 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:26.584659 (+ 18527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:26.584667 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:26.584706 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:26.585212 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:26.585227 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.587139 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.587146 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:26.589000 (+ 1854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:26.589014 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.590891 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.590898 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:26.591069 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:26.591106 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 00:58:26.591175 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 00:58:26.591307 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":5659,"threads_started":1}
I20260810 00:58:26.592808 29283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:26.566472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40694 (local address 127.20.81.130:36539)
0810 00:58:26.584035 (+ 17563us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:26.584042 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:26.584065 (+ 23us) server_negotiation.cc:407] Connection header received
0810 00:58:26.584155 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:26.584160 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:26.584419 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:26.584664 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:26.585371 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.586966 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.589217 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.590680 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.592041 (+ 1361us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:26.592257 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:26.592336 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 00:58:26.592509 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17418,"thread_start_us":17255,"threads_started":1}
I20260810 00:58:26.593967 29281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:26.566317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:33253 (local address 127.0.0.1:55072)
0810 00:58:26.585351 (+ 19034us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:26.585369 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:26.585501 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:26.586216 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:26.586222 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:26.586247 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:26.586559 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:26.586571 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.588430 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.588436 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:26.590544 (+ 2108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:26.590559 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.593225 (+ 2666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.593231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:26.593393 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:26.593423 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 00:58:26.593498 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 00:58:26.593625 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18877,"thread_start_us":16819,"threads_started":1}
I20260810 00:58:26.594710 29284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:26.566955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55072 (local address 127.20.81.129:33253)
0810 00:58:26.581321 (+ 14366us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:26.581331 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:26.585594 (+ 4263us) server_negotiation.cc:407] Connection header received
0810 00:58:26.585724 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:26.585734 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:26.586005 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:26.586214 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:26.586765 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.588275 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.590749 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.592702 (+ 1953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.592838 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:26.593082 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:26.594164 (+ 1082us) server_negotiation.cc:299] Negotiation successful
0810 00:58:26.594362 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14148,"thread_start_us":15607,"threads_started":1}
I20260810 00:58:26.596810 29017 tablet_service.cc:1467] Processing CreateTablet for tablet f179228afae341a2923f9411539cf30d (DEFAULT_TABLE table=test-table [id=76fdd90a443f4485b96cf97d157102bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:26.598939 29017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f179228afae341a2923f9411539cf30d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:26.599763 29282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:26.566059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:35525 (local address 127.0.0.1:49732)
0810 00:58:26.583414 (+ 17355us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:26.583434 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:26.583552 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:26.586887 (+ 3335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:26.586894 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:26.586922 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:26.587416 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:26.587427 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.595953 (+ 8526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.595959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:26.597238 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:26.597249 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.598945 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.598953 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:26.599140 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:26.599177 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 00:58:26.599292 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 00:58:26.599439 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17220,"thread_start_us":18984,"threads_started":1}
I20260810 00:58:26.599848 29285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:26.567045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49732 (local address 127.20.81.131:35525)
0810 00:58:26.586084 (+ 19039us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:26.586092 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:26.586127 (+ 35us) server_negotiation.cc:407] Connection header received
0810 00:58:26.586228 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:26.586236 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:26.586504 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:26.586864 (+ 360us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:26.594685 (+ 7821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.595802 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.597396 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.598731 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.598879 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:26.599133 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:26.599370 (+ 237us) server_negotiation.cc:299] Negotiation successful
0810 00:58:26.599530 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18861,"thread_start_us":18702,"threads_started":1}
I20260810 00:58:26.603503 29175 tablet_service.cc:1467] Processing CreateTablet for tablet f179228afae341a2923f9411539cf30d (DEFAULT_TABLE table=test-table [id=76fdd90a443f4485b96cf97d157102bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:26.603504 28851 tablet_service.cc:1467] Processing CreateTablet for tablet f179228afae341a2923f9411539cf30d (DEFAULT_TABLE table=test-table [id=76fdd90a443f4485b96cf97d157102bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:26.604853 29175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f179228afae341a2923f9411539cf30d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:26.604884 28851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f179228afae341a2923f9411539cf30d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:26.611891 29286 tablet_bootstrap.cc:492] T f179228afae341a2923f9411539cf30d P 4b439751b0f64ffd8322d2894db470ac: Bootstrap starting.
I20260810 00:58:26.614892 29286 tablet_bootstrap.cc:654] T f179228afae341a2923f9411539cf30d P 4b439751b0f64ffd8322d2894db470ac: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:26.615868 29286 log.cc:824] T f179228afae341a2923f9411539cf30d P 4b439751b0f64ffd8322d2894db470ac: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:26.619306 29286 tablet_bootstrap.cc:492] T f179228afae341a2923f9411539cf30d P 4b439751b0f64ffd8322d2894db470ac: No bootstrap required, opened a new log
I20260810 00:58:26.619614 29286 ts_tablet_manager.cc:1397] T f179228afae341a2923f9411539cf30d P 4b439751b0f64ffd8322d2894db470ac: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 00:58:26.620432 29288 tablet_bootstrap.cc:492] T f179228afae341a2923f9411539cf30d P 64786d2c70b344f8954cb3344937e3b7: Bootstrap starting.
I20260810 00:58:26.621024 29289 tablet_bootstrap.cc:492] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07: Bootstrap starting.
I20260810 00:58:26.624547 29289 tablet_bootstrap.cc:654] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:26.624985 29288 tablet_bootstrap.cc:654] T f179228afae341a2923f9411539cf30d P 64786d2c70b344f8954cb3344937e3b7: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:26.625777 29289 log.cc:824] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:26.626021 29288 log.cc:824] T f179228afae341a2923f9411539cf30d P 64786d2c70b344f8954cb3344937e3b7: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:26.629505 29289 tablet_bootstrap.cc:492] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07: No bootstrap required, opened a new log
I20260810 00:58:26.629784 29289 ts_tablet_manager.cc:1397] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260810 00:58:26.630024 29286 raft_consensus.cc:348] T f179228afae341a2923f9411539cf30d P 4b439751b0f64ffd8322d2894db470ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b439751b0f64ffd8322d2894db470ac" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36539 } } peers { permanent_uuid: "94f458e21af3447e8489ca2b0f0c2e07" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35525 } } peers { permanent_uuid: "64786d2c70b344f8954cb3344937e3b7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33253 } }
I20260810 00:58:26.630563 29286 raft_consensus.cc:374] T f179228afae341a2923f9411539cf30d P 4b439751b0f64ffd8322d2894db470ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:26.630725 29286 raft_consensus.cc:729] T f179228afae341a2923f9411539cf30d P 4b439751b0f64ffd8322d2894db470ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b439751b0f64ffd8322d2894db470ac, State: Initialized, Role: FOLLOWER
I20260810 00:58:26.631376 29286 consensus_queue.cc:260] T f179228afae341a2923f9411539cf30d P 4b439751b0f64ffd8322d2894db470ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b439751b0f64ffd8322d2894db470ac" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36539 } } peers { permanent_uuid: "94f458e21af3447e8489ca2b0f0c2e07" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35525 } } peers { permanent_uuid: "64786d2c70b344f8954cb3344937e3b7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33253 } }
I20260810 00:58:26.632578 29095 heartbeater.cc:503] Master 127.20.81.190:33681 was elected leader, sending a full tablet report...
I20260810 00:58:26.633297 29286 ts_tablet_manager.cc:1428] T f179228afae341a2923f9411539cf30d P 4b439751b0f64ffd8322d2894db470ac: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260810 00:58:26.633545 29288 tablet_bootstrap.cc:492] T f179228afae341a2923f9411539cf30d P 64786d2c70b344f8954cb3344937e3b7: No bootstrap required, opened a new log
I20260810 00:58:26.633795 29288 ts_tablet_manager.cc:1397] T f179228afae341a2923f9411539cf30d P 64786d2c70b344f8954cb3344937e3b7: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.005s
I20260810 00:58:26.641119 29289 raft_consensus.cc:348] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b439751b0f64ffd8322d2894db470ac" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36539 } } peers { permanent_uuid: "94f458e21af3447e8489ca2b0f0c2e07" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35525 } } peers { permanent_uuid: "64786d2c70b344f8954cb3344937e3b7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33253 } }
I20260810 00:58:26.641458 29289 raft_consensus.cc:374] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:26.641582 29289 raft_consensus.cc:729] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94f458e21af3447e8489ca2b0f0c2e07, State: Initialized, Role: FOLLOWER
I20260810 00:58:26.642125 29289 consensus_queue.cc:260] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b439751b0f64ffd8322d2894db470ac" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36539 } } peers { permanent_uuid: "94f458e21af3447e8489ca2b0f0c2e07" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35525 } } peers { permanent_uuid: "64786d2c70b344f8954cb3344937e3b7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33253 } }
I20260810 00:58:26.643257 29249 heartbeater.cc:503] Master 127.20.81.190:33681 was elected leader, sending a full tablet report...
I20260810 00:58:26.643870 29289 ts_tablet_manager.cc:1428] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07: Time spent starting tablet: real 0.014s user 0.007s sys 0.006s
I20260810 00:58:26.645361 29288 raft_consensus.cc:348] T f179228afae341a2923f9411539cf30d P 64786d2c70b344f8954cb3344937e3b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b439751b0f64ffd8322d2894db470ac" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36539 } } peers { permanent_uuid: "94f458e21af3447e8489ca2b0f0c2e07" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35525 } } peers { permanent_uuid: "64786d2c70b344f8954cb3344937e3b7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33253 } }
I20260810 00:58:26.646220 29288 raft_consensus.cc:374] T f179228afae341a2923f9411539cf30d P 64786d2c70b344f8954cb3344937e3b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:26.646391 29288 raft_consensus.cc:729] T f179228afae341a2923f9411539cf30d P 64786d2c70b344f8954cb3344937e3b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64786d2c70b344f8954cb3344937e3b7, State: Initialized, Role: FOLLOWER
I20260810 00:58:26.647032 29288 consensus_queue.cc:260] T f179228afae341a2923f9411539cf30d P 64786d2c70b344f8954cb3344937e3b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b439751b0f64ffd8322d2894db470ac" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36539 } } peers { permanent_uuid: "94f458e21af3447e8489ca2b0f0c2e07" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35525 } } peers { permanent_uuid: "64786d2c70b344f8954cb3344937e3b7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33253 } }
I20260810 00:58:26.649115 29288 ts_tablet_manager.cc:1428] T f179228afae341a2923f9411539cf30d P 64786d2c70b344f8954cb3344937e3b7: Time spent starting tablet: real 0.015s user 0.012s sys 0.000s
W20260810 00:58:26.716962 29251 tablet.cc:2369] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:58:26.740249 29097 tablet.cc:2369] T f179228afae341a2923f9411539cf30d P 4b439751b0f64ffd8322d2894db470ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:58:26.760076 29294 raft_consensus.cc:482] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:58:26.760394 29294 raft_consensus.cc:504] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b439751b0f64ffd8322d2894db470ac" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36539 } } peers { permanent_uuid: "94f458e21af3447e8489ca2b0f0c2e07" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35525 } } peers { permanent_uuid: "64786d2c70b344f8954cb3344937e3b7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33253 } }
I20260810 00:58:26.762126 29294 leader_election.cc:288] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b439751b0f64ffd8322d2894db470ac (127.20.81.130:36539), 64786d2c70b344f8954cb3344937e3b7 (127.20.81.129:33253)
I20260810 00:58:26.767763 29283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:26.762667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:59635 (local address 127.20.81.130:36539)
0810 00:58:26.762903 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:26.762919 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:26.762942 (+ 23us) server_negotiation.cc:407] Connection header received
0810 00:58:26.763017 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:26.763023 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:26.763105 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:26.763218 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:26.763590 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.764314 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.766039 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.766888 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.767067 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:26.767142 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:26.767387 (+ 245us) server_negotiation.cc:299] Negotiation successful
0810 00:58:26.767508 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260810 00:58:26.767802 29255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:26.762350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:36539 (local address 127.20.81.131:59635)
0810 00:58:26.762632 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:26.762659 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:26.762801 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:26.763236 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:26.763241 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:26.763264 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:26.763450 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:26.763460 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.764451 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.764457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:26.765894 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:26.765906 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.767038 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.767043 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:26.767227 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:26.767251 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 00:58:26.767359 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 00:58:26.767487 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":34}
I20260810 00:58:26.768461 29284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:26.762670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:35787 (local address 127.20.81.129:33253)
0810 00:58:26.763124 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:26.763131 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:26.763152 (+ 21us) server_negotiation.cc:407] Connection header received
0810 00:58:26.763235 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:26.763240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:26.763300 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:26.763414 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:26.763827 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.764562 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.766537 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.767396 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.767584 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:26.767659 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:26.768123 (+ 464us) server_negotiation.cc:299] Negotiation successful
0810 00:58:26.768257 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295}
I20260810 00:58:26.768489 29256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:26.762580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:33253 (local address 127.20.81.131:35787)
0810 00:58:26.762882 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:26.762900 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:26.763007 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:26.763438 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:26.763443 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:26.763468 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:26.763650 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:26.763659 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.764709 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.764716 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:26.766307 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:26.766320 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.767603 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.767610 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:26.767896 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:26.767950 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 00:58:26.768050 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 00:58:26.768184 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163}
I20260810 00:58:26.768903 29039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f179228afae341a2923f9411539cf30d" candidate_uuid: "94f458e21af3447e8489ca2b0f0c2e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b439751b0f64ffd8322d2894db470ac" is_pre_election: true
I20260810 00:58:26.769493 29039 raft_consensus.cc:2454] T f179228afae341a2923f9411539cf30d P 4b439751b0f64ffd8322d2894db470ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94f458e21af3447e8489ca2b0f0c2e07 in term 0.
I20260810 00:58:26.769608 28873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f179228afae341a2923f9411539cf30d" candidate_uuid: "94f458e21af3447e8489ca2b0f0c2e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64786d2c70b344f8954cb3344937e3b7" is_pre_election: true
I20260810 00:58:26.770155 28873 raft_consensus.cc:2454] T f179228afae341a2923f9411539cf30d P 64786d2c70b344f8954cb3344937e3b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94f458e21af3447e8489ca2b0f0c2e07 in term 0.
I20260810 00:58:26.770223 29126 leader_election.cc:302] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b439751b0f64ffd8322d2894db470ac, 94f458e21af3447e8489ca2b0f0c2e07; no voters:
I20260810 00:58:26.770702 29294 raft_consensus.cc:2781] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:58:26.770922 29294 raft_consensus.cc:482] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:58:26.771085 29294 raft_consensus.cc:3037] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:26.773496 29294 raft_consensus.cc:504] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b439751b0f64ffd8322d2894db470ac" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36539 } } peers { permanent_uuid: "94f458e21af3447e8489ca2b0f0c2e07" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35525 } } peers { permanent_uuid: "64786d2c70b344f8954cb3344937e3b7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33253 } }
I20260810 00:58:26.774176 29294 leader_election.cc:288] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07 [CANDIDATE]: Term 1 election: Requested vote from peers 4b439751b0f64ffd8322d2894db470ac (127.20.81.130:36539), 64786d2c70b344f8954cb3344937e3b7 (127.20.81.129:33253)
I20260810 00:58:26.774673 29039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f179228afae341a2923f9411539cf30d" candidate_uuid: "94f458e21af3447e8489ca2b0f0c2e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b439751b0f64ffd8322d2894db470ac"
I20260810 00:58:26.774727 28873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f179228afae341a2923f9411539cf30d" candidate_uuid: "94f458e21af3447e8489ca2b0f0c2e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64786d2c70b344f8954cb3344937e3b7"
I20260810 00:58:26.774926 29039 raft_consensus.cc:3037] T f179228afae341a2923f9411539cf30d P 4b439751b0f64ffd8322d2894db470ac [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:26.774926 28873 raft_consensus.cc:3037] T f179228afae341a2923f9411539cf30d P 64786d2c70b344f8954cb3344937e3b7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:26.778645 28873 raft_consensus.cc:2454] T f179228afae341a2923f9411539cf30d P 64786d2c70b344f8954cb3344937e3b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94f458e21af3447e8489ca2b0f0c2e07 in term 1.
I20260810 00:58:26.778645 29039 raft_consensus.cc:2454] T f179228afae341a2923f9411539cf30d P 4b439751b0f64ffd8322d2894db470ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94f458e21af3447e8489ca2b0f0c2e07 in term 1.
I20260810 00:58:26.779127 29124 leader_election.cc:302] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64786d2c70b344f8954cb3344937e3b7, 94f458e21af3447e8489ca2b0f0c2e07; no voters:
I20260810 00:58:26.779522 29294 raft_consensus.cc:2781] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:58:26.779971 29294 raft_consensus.cc:686] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07 [term 1 LEADER]: Becoming Leader. State: Replica: 94f458e21af3447e8489ca2b0f0c2e07, State: Running, Role: LEADER
I20260810 00:58:26.780500 29294 consensus_queue.cc:237] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b439751b0f64ffd8322d2894db470ac" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36539 } } peers { permanent_uuid: "94f458e21af3447e8489ca2b0f0c2e07" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35525 } } peers { permanent_uuid: "64786d2c70b344f8954cb3344937e3b7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33253 } }
I20260810 00:58:26.786092 27838 catalog_manager.cc:5495] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94f458e21af3447e8489ca2b0f0c2e07 (127.20.81.131). New cstate: current_term: 1 leader_uuid: "94f458e21af3447e8489ca2b0f0c2e07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b439751b0f64ffd8322d2894db470ac" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94f458e21af3447e8489ca2b0f0c2e07" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64786d2c70b344f8954cb3344937e3b7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33253 } health_report { overall_health: UNKNOWN } } }
W20260810 00:58:26.815833 28925 tablet.cc:2369] T f179228afae341a2923f9411539cf30d P 64786d2c70b344f8954cb3344937e3b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:58:26.815995 29285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:26.810623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49736 (local address 127.20.81.131:35525)
0810 00:58:26.810843 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:26.810851 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:26.811015 (+ 164us) server_negotiation.cc:407] Connection header received
0810 00:58:26.811186 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:26.811190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:26.811278 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:26.811412 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:58:26.811737 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.812458 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.813848 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.814262 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.814372 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:26.814882 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:58:26.815399 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:58:26.815551 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:26.815608 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 00:58:26.815695 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 00:58:26.824385 28873 raft_consensus.cc:1261] T f179228afae341a2923f9411539cf30d P 64786d2c70b344f8954cb3344937e3b7 [term 1 FOLLOWER]: Refusing update from remote peer 94f458e21af3447e8489ca2b0f0c2e07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:58:26.824594 29039 raft_consensus.cc:1261] T f179228afae341a2923f9411539cf30d P 4b439751b0f64ffd8322d2894db470ac [term 1 FOLLOWER]: Refusing update from remote peer 94f458e21af3447e8489ca2b0f0c2e07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:58:26.825501 29297 consensus_queue.cc:1035] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64786d2c70b344f8954cb3344937e3b7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:58:26.825847 29294 consensus_queue.cc:1035] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b439751b0f64ffd8322d2894db470ac" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:58:26.839527 29301 mvcc.cc:205] Tried to move back new op lower bound from 7316781083942174720 to 7316781083776958464. Current Snapshot: MvccSnapshot[applied={T|T < 7316781083942174720}]
I20260810 00:58:26.849550 29302 mvcc.cc:205] Tried to move back new op lower bound from 7316781083942174720 to 7316781083776958464. Current Snapshot: MvccSnapshot[applied={T|T < 7316781083942174720}]
I20260810 00:58:26.864185 27838 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:58:26.864605 27838 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:58:26.879166 27838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 31ff0eea6672410291c03a7df56d1cf2: Sending DeleteTablet for 3 replicas of tablet f179228afae341a2923f9411539cf30d
I20260810 00:58:26.880622 29017 tablet_service.cc:1514] Processing DeleteTablet for tablet f179228afae341a2923f9411539cf30d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:26 UTC) from {username='oryx', principal='oryx/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:40694
I20260810 00:58:26.880839 29175 tablet_service.cc:1514] Processing DeleteTablet for tablet f179228afae341a2923f9411539cf30d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:26 UTC) from {username='oryx', principal='oryx/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:49732
I20260810 00:58:26.882507 29313 tablet_replica.cc:330] T f179228afae341a2923f9411539cf30d P 4b439751b0f64ffd8322d2894db470ac: stopping tablet replica
I20260810 00:58:26.882552 29174 tablet_service.cc:1514] Processing DeleteTablet for tablet f179228afae341a2923f9411539cf30d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:26 UTC) from {username='oryx', principal='oryx/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:49732
I20260810 00:58:26.883178 29313 raft_consensus.cc:2229] T f179228afae341a2923f9411539cf30d P 4b439751b0f64ffd8322d2894db470ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:58:26.884096 28851 tablet_service.cc:1514] Processing DeleteTablet for tablet f179228afae341a2923f9411539cf30d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:26 UTC) from {username='oryx', principal='oryx/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:55072
I20260810 00:58:26.884246 29313 raft_consensus.cc:2258] T f179228afae341a2923f9411539cf30d P 4b439751b0f64ffd8322d2894db470ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:26.893050 29313 ts_tablet_manager.cc:1905] T f179228afae341a2923f9411539cf30d P 4b439751b0f64ffd8322d2894db470ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:26.899830 29316 tablet_replica.cc:330] T f179228afae341a2923f9411539cf30d P 64786d2c70b344f8954cb3344937e3b7: stopping tablet replica
I20260810 00:58:26.900573 29316 raft_consensus.cc:2229] T f179228afae341a2923f9411539cf30d P 64786d2c70b344f8954cb3344937e3b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:58:26.900775 29315 tablet_replica.cc:330] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07: stopping tablet replica
I20260810 00:58:26.901535 29316 raft_consensus.cc:2258] T f179228afae341a2923f9411539cf30d P 64786d2c70b344f8954cb3344937e3b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:26.901533 29315 raft_consensus.cc:2229] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:58:26.902593 29315 raft_consensus.cc:2258] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:26.904176 29316 ts_tablet_manager.cc:1905] T f179228afae341a2923f9411539cf30d P 64786d2c70b344f8954cb3344937e3b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:26.905419 29315 ts_tablet_manager.cc:1905] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:26.907908 29313 ts_tablet_manager.cc:1918] T f179228afae341a2923f9411539cf30d P 4b439751b0f64ffd8322d2894db470ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:58:26.908221 29313 log.cc:1196] T f179228afae341a2923f9411539cf30d P 4b439751b0f64ffd8322d2894db470ac: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/f179228afae341a2923f9411539cf30d
I20260810 00:58:26.909332 29313 ts_tablet_manager.cc:1939] T f179228afae341a2923f9411539cf30d P 4b439751b0f64ffd8322d2894db470ac: Deleting consensus metadata
I20260810 00:58:26.911846 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 27803
I20260810 00:58:26.913172 27825 catalog_manager.cc:4841] TS 4b439751b0f64ffd8322d2894db470ac (127.20.81.130:36539): tablet f179228afae341a2923f9411539cf30d (table test-table [id=76fdd90a443f4485b96cf97d157102bb]) successfully deleted
I20260810 00:58:26.913918 27869 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:26.915100 29316 ts_tablet_manager.cc:1918] T f179228afae341a2923f9411539cf30d P 64786d2c70b344f8954cb3344937e3b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:58:26.915381 29316 log.cc:1196] T f179228afae341a2923f9411539cf30d P 64786d2c70b344f8954cb3344937e3b7: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/wal/wals/f179228afae341a2923f9411539cf30d
I20260810 00:58:26.916877 29316 ts_tablet_manager.cc:1939] T f179228afae341a2923f9411539cf30d P 64786d2c70b344f8954cb3344937e3b7: Deleting consensus metadata
I20260810 00:58:26.924677 29315 ts_tablet_manager.cc:1918] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:58:26.924960 29315 log.cc:1196] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/f179228afae341a2923f9411539cf30d
I20260810 00:58:26.925843 29315 ts_tablet_manager.cc:1939] T f179228afae341a2923f9411539cf30d P 94f458e21af3447e8489ca2b0f0c2e07: Deleting consensus metadata
I20260810 00:58:26.939230 29253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:26.889416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41502 (local address 127.20.81.189:46407)
0810 00:58:26.889647 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:26.889657 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:26.912489 (+ 22832us) server_negotiation.cc:407] Connection header received
0810 00:58:26.912835 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:26.912843 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:26.912944 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:26.913066 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:26.918181 (+ 5115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.919620 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.925159 (+ 5539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.926260 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.926395 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:26.932342 (+ 5947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:26.932347 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:26.932350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:26.932387 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:26.935902 (+ 3515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:26.936655 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:26.936660 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:26.936662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:26.936820 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:26.937241 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:26.937249 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:26.937253 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:26.937655 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:26.937972 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:26.938325 (+ 353us) server_negotiation.cc:299] Negotiation successful
0810 00:58:26.938569 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 00:58:26.957232 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:26.910472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48590 (local address 127.20.81.188:45483)
0810 00:58:26.910676 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:26.910682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:26.939340 (+ 28658us) server_negotiation.cc:407] Connection header received
0810 00:58:26.939612 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:26.939618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:26.939717 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:26.939811 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:26.942336 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.943292 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.944938 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:26.945643 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:26.945813 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:26.949134 (+ 3321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:26.949139 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:26.949142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:26.949174 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:26.953246 (+ 4072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:26.954112 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:26.954120 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:26.954123 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:26.954332 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:26.955210 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:26.955222 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:26.955225 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:26.955643 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:26.956124 (+ 481us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:26.956421 (+ 297us) server_negotiation.cc:299] Negotiation successful
0810 00:58:26.956647 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 00:58:27.107193 27823 catalog_manager.cc:4841] TS 64786d2c70b344f8954cb3344937e3b7 (127.20.81.129:33253): tablet f179228afae341a2923f9411539cf30d (table test-table [id=76fdd90a443f4485b96cf97d157102bb]) successfully deleted
I20260810 00:58:27.108031 27824 catalog_manager.cc:4841] TS 94f458e21af3447e8489ca2b0f0c2e07 (127.20.81.131:35525): tablet f179228afae341a2923f9411539cf30d (table test-table [id=76fdd90a443f4485b96cf97d157102bb]) successfully deleted
I20260810 00:58:27.108108 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 27803
W20260810 00:58:27.133014 29123 connection.cc:535] server connection from 127.0.0.1:49732 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 00:58:27.156980 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:27.139084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48598 (local address 127.20.81.188:45483)
0810 00:58:27.139298 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:27.139306 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:27.139945 (+ 639us) server_negotiation.cc:407] Connection header received
0810 00:58:27.140159 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:27.140165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:27.140259 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:27.140378 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:27.141991 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:27.142966 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:27.144917 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:27.145570 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:27.145706 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:27.150693 (+ 4987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:27.150698 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:27.150701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:27.150733 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:27.153807 (+ 3074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:27.154586 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:27.154593 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:27.154597 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:27.154763 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:27.155271 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:27.155279 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:27.155284 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:27.155730 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:27.155994 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:27.156364 (+ 370us) server_negotiation.cc:299] Negotiation successful
0810 00:58:27.156621 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 00:58:27.158279 29253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:27.138056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41508 (local address 127.20.81.189:46407)
0810 00:58:27.138319 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:27.138329 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:27.139405 (+ 1076us) server_negotiation.cc:407] Connection header received
0810 00:58:27.140330 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:27.140336 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:27.140426 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:27.140536 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:27.142698 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:27.143800 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:27.145726 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:27.146478 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:27.146611 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:27.150513 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:27.150520 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:27.150525 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:27.150568 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:27.154537 (+ 3969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:27.155432 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:27.155439 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:27.155443 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:27.155588 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:27.156046 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:27.156051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:27.156054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:27.156529 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:27.156997 (+ 468us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:27.157553 (+ 556us) server_negotiation.cc:299] Negotiation successful
0810 00:58:27.157778 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 00:58:27.859402 29327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:27.858656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.129:55381)
0810 00:58:27.859086 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:27.859223 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":213,"threads_started":1}
W20260810 00:58:27.859889 28923 heartbeater.cc:644] Failed to heartbeat to 127.20.81.190:33681 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
I20260810 00:58:27.882025 29328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:27.881296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.131:57139)
0810 00:58:27.881726 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:27.881851 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":238,"threads_started":1}
W20260810 00:58:27.882547 29249 heartbeater.cc:644] Failed to heartbeat to 127.20.81.190:33681 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
I20260810 00:58:27.882875 29329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:27.881805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.130:59631)
0810 00:58:27.882570 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:27.882716 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":258,"threads_started":1}
W20260810 00:58:27.883242 29095 heartbeater.cc:644] Failed to heartbeat to 127.20.81.190:33681 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
I20260810 00:58:28.377811 29330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31ff0eea6672410291c03a7df56d1cf2)
I20260810 00:58:28.378123 29330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: false } }
I20260810 00:58:28.379135 29331 raft_consensus.cc:482] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31ff0eea6672410291c03a7df56d1cf2)
I20260810 00:58:28.379369 29331 raft_consensus.cc:504] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: false } }
I20260810 00:58:28.379704 29330 leader_election.cc:288] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31ff0eea6672410291c03a7df56d1cf2 (127.20.81.190:33681), 025dbc759c9a4ed3986536b4df6c761d (127.20.81.189:46407)
I20260810 00:58:28.386819 29331 leader_election.cc:288] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31ff0eea6672410291c03a7df56d1cf2 (127.20.81.190:33681), 6f8bda17b7544088b413ba1a962fdd6d (127.20.81.188:45483)
I20260810 00:58:28.390265 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:28.380447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.0.0.1:46786)
0810 00:58:28.389784 (+ 9337us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:28.389984 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9194,"threads_started":1}
W20260810 00:58:28.391572 28494 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
I20260810 00:58:28.394440 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:28.379899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.0.0.1:46770)
0810 00:58:28.392443 (+ 12544us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:28.392747 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12398,"threads_started":1}
I20260810 00:58:28.399098 29339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:28.394886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.0.0.1:46814)
0810 00:58:28.398571 (+ 3685us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:28.398806 (+ 235us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3567,"thread_start_us":3355,"threads_started":1}
W20260810 00:58:28.399078 28708 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
W20260810 00:58:28.399326 28494 leader_election.cc:334] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 31ff0eea6672410291c03a7df56d1cf2 (127.20.81.190:33681): Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
I20260810 00:58:28.401155 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:28.400491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.0.0.1:46822)
0810 00:58:28.400682 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:28.400894 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260810 00:58:28.401378 28708 leader_election.cc:334] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 31ff0eea6672410291c03a7df56d1cf2 (127.20.81.190:33681): Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
I20260810 00:58:28.402794 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:28.391231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:45483 (local address 127.0.0.1:48604)
0810 00:58:28.392181 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:28.392202 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:28.392324 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:28.395235 (+ 2911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:28.395243 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:28.395309 (+ 66us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:28.395785 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:28.395802 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:28.397998 (+ 2196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:28.398007 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:28.400296 (+ 2289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:28.400313 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:28.401968 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:28.401974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:28.402150 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:28.402174 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 00:58:28.402281 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 00:58:28.402482 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"mutex_wait_us":57,"spinlock_wait_cycles":10496}
I20260810 00:58:28.402845 29336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:28.391260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48604 (local address 127.20.81.188:45483)
0810 00:58:28.394571 (+ 3311us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:28.394581 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:28.394607 (+ 26us) server_negotiation.cc:407] Connection header received
0810 00:58:28.394692 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:28.394700 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:28.394797 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:28.394921 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:28.396019 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:28.397792 (+ 1773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:28.400468 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:28.401789 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:28.401945 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:28.402038 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:28.402321 (+ 283us) server_negotiation.cc:299] Negotiation successful
0810 00:58:28.402496 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3119,"thread_start_us":2935,"threads_started":1}
I20260810 00:58:28.403192 29333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:28.380040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41518 (local address 127.20.81.189:46407)
0810 00:58:28.391027 (+ 10987us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:28.391032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:28.394881 (+ 3849us) server_negotiation.cc:407] Connection header received
0810 00:58:28.395168 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:28.395177 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:28.395300 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:28.395386 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:28.397030 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:28.398701 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:28.401185 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:28.402085 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:28.402226 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:28.402335 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:28.402651 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 00:58:28.402789 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":10851,"thread_start_us":9213,"threads_started":1}
I20260810 00:58:28.403440 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:28.380103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:46407 (local address 127.0.0.1:41518)
0810 00:58:28.394780 (+ 14677us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:28.394799 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:28.394981 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:28.396303 (+ 1322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:28.396315 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:28.396360 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:28.396794 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:28.396811 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:28.398886 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:28.398891 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:28.401024 (+ 2133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:28.401039 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:28.402255 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:28.402261 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:28.402491 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:28.402513 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 00:58:28.402615 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 00:58:28.402761 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14567,"mutex_wait_us":2,"thread_start_us":12061,"threads_started":1}
I20260810 00:58:28.403609 28734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "025dbc759c9a4ed3986536b4df6c761d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6f8bda17b7544088b413ba1a962fdd6d" is_pre_election: true
I20260810 00:58:28.403983 28734 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 025dbc759c9a4ed3986536b4df6c761d in term 1.
I20260810 00:58:28.404139 28519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f8bda17b7544088b413ba1a962fdd6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "025dbc759c9a4ed3986536b4df6c761d" is_pre_election: true
I20260810 00:58:28.404525 28519 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8bda17b7544088b413ba1a962fdd6d in term 1.
I20260810 00:58:28.404577 28494 leader_election.cc:302] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 025dbc759c9a4ed3986536b4df6c761d, 6f8bda17b7544088b413ba1a962fdd6d; no voters: 31ff0eea6672410291c03a7df56d1cf2
I20260810 00:58:28.404934 29331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 00:58:28.405077 29331 raft_consensus.cc:482] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 31ff0eea6672410291c03a7df56d1cf2)
I20260810 00:58:28.405040 28705 leader_election.cc:302] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 025dbc759c9a4ed3986536b4df6c761d, 6f8bda17b7544088b413ba1a962fdd6d; no voters: 31ff0eea6672410291c03a7df56d1cf2
I20260810 00:58:28.405189 29331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 1 FOLLOWER]: Advancing to term 2
I20260810 00:58:28.405473 29330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 00:58:28.405618 29330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 31ff0eea6672410291c03a7df56d1cf2)
I20260810 00:58:28.405738 29330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 1 FOLLOWER]: Advancing to term 2
I20260810 00:58:28.407598 29331 raft_consensus.cc:504] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: false } }
I20260810 00:58:28.407972 29330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: false } }
I20260810 00:58:28.408152 29331 leader_election.cc:288] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [CANDIDATE]: Term 2 election: Requested vote from peers 31ff0eea6672410291c03a7df56d1cf2 (127.20.81.190:33681), 6f8bda17b7544088b413ba1a962fdd6d (127.20.81.188:45483)
I20260810 00:58:28.408675 29330 leader_election.cc:288] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [CANDIDATE]: Term 2 election: Requested vote from peers 31ff0eea6672410291c03a7df56d1cf2 (127.20.81.190:33681), 025dbc759c9a4ed3986536b4df6c761d (127.20.81.189:46407)
I20260810 00:58:28.408826 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:28.408350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.0.0.1:46830)
0810 00:58:28.408487 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:28.408590 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 00:58:28.408879 28734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "025dbc759c9a4ed3986536b4df6c761d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6f8bda17b7544088b413ba1a962fdd6d"
I20260810 00:58:28.409153 28734 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 025dbc759c9a4ed3986536b4df6c761d in current term 2: Already voted for candidate 6f8bda17b7544088b413ba1a962fdd6d in this term.
I20260810 00:58:28.409226 28519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f8bda17b7544088b413ba1a962fdd6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "025dbc759c9a4ed3986536b4df6c761d"
I20260810 00:58:28.409636 28519 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6f8bda17b7544088b413ba1a962fdd6d in current term 2: Already voted for candidate 025dbc759c9a4ed3986536b4df6c761d in this term.
I20260810 00:58:28.410219 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:28.409918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.0.0.1:46848)
0810 00:58:28.410059 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:28.410132 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 00:58:28.410362 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:28.408762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.0.0.1:46842)
0810 00:58:28.410130 (+ 1368us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:28.410224 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1263}
W20260810 00:58:28.410571 28494 leader_election.cc:334] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 31ff0eea6672410291c03a7df56d1cf2 (127.20.81.190:33681): Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
I20260810 00:58:28.410763 28494 leader_election.cc:302] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 025dbc759c9a4ed3986536b4df6c761d; no voters: 31ff0eea6672410291c03a7df56d1cf2, 6f8bda17b7544088b413ba1a962fdd6d
I20260810 00:58:28.411074 29331 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 00:58:28.411310 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:28.410953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.0.0.1:46850)
0810 00:58:28.411076 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:28.411189 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 00:58:28.411505 28708 leader_election.cc:334] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 31ff0eea6672410291c03a7df56d1cf2 (127.20.81.190:33681): Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
I20260810 00:58:28.411638 28708 leader_election.cc:302] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8bda17b7544088b413ba1a962fdd6d; no voters: 025dbc759c9a4ed3986536b4df6c761d, 31ff0eea6672410291c03a7df56d1cf2
I20260810 00:58:28.411909 29330 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 00:58:28.861521 29341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:28.860625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.129:38865)
0810 00:58:28.861134 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:28.861248 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":256,"threads_started":1}
I20260810 00:58:28.884109 29343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:28.883274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.131:39453)
0810 00:58:28.883849 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:28.883952 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":312,"threads_started":1}
I20260810 00:58:28.884642 29344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:28.883900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.130:58571)
0810 00:58:28.884399 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:28.884478 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":294,"threads_started":1}
I20260810 00:58:29.863289 29345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:29.862535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.129:42691)
0810 00:58:29.862961 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:29.863068 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":213,"threads_started":1}
I20260810 00:58:29.885835 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:29.885024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.131:42657)
0810 00:58:29.885539 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:29.885632 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":294,"threads_started":1}
I20260810 00:58:29.886199 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:29.885437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.130:57433)
0810 00:58:29.885935 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:29.886050 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":294,"threads_started":1}
I20260810 00:58:30.126260 29348 raft_consensus.cc:482] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:58:30.126549 29348 raft_consensus.cc:504] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: false } }
I20260810 00:58:30.127199 29348 leader_election.cc:288] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31ff0eea6672410291c03a7df56d1cf2 (127.20.81.190:33681), 6f8bda17b7544088b413ba1a962fdd6d (127.20.81.188:45483)
I20260810 00:58:30.128386 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.127409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.0.0.1:47128)
0810 00:58:30.127995 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.128126 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":292,"threads_started":1}
I20260810 00:58:30.128371 28734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "025dbc759c9a4ed3986536b4df6c761d" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6f8bda17b7544088b413ba1a962fdd6d" is_pre_election: true
I20260810 00:58:30.128660 28734 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 025dbc759c9a4ed3986536b4df6c761d in term 2.
I20260810 00:58:30.129220 28494 leader_election.cc:302] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025dbc759c9a4ed3986536b4df6c761d, 6f8bda17b7544088b413ba1a962fdd6d; no voters:
I20260810 00:58:30.129590 29348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260810 00:58:30.129694 29348 raft_consensus.cc:482] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:58:30.129887 29348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 2 FOLLOWER]: Advancing to term 3
I20260810 00:58:30.130209 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.129766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.0.0.1:47130)
0810 00:58:30.129960 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.130048 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260810 00:58:30.130515 28494 leader_election.cc:334] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 31ff0eea6672410291c03a7df56d1cf2 (127.20.81.190:33681): Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
I20260810 00:58:30.132457 29348 raft_consensus.cc:504] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: false } }
I20260810 00:58:30.133019 29348 leader_election.cc:288] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [CANDIDATE]: Term 3 election: Requested vote from peers 31ff0eea6672410291c03a7df56d1cf2 (127.20.81.190:33681), 6f8bda17b7544088b413ba1a962fdd6d (127.20.81.188:45483)
I20260810 00:58:30.133509 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.133198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.0.0.1:47134)
0810 00:58:30.133338 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.133400 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 00:58:30.133867 28734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "025dbc759c9a4ed3986536b4df6c761d" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6f8bda17b7544088b413ba1a962fdd6d"
I20260810 00:58:30.134109 28734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 2 FOLLOWER]: Advancing to term 3
I20260810 00:58:30.134603 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.134123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.0.0.1:47138)
0810 00:58:30.134307 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.134444 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 00:58:30.134763 28494 leader_election.cc:334] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 31ff0eea6672410291c03a7df56d1cf2 (127.20.81.190:33681): Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
I20260810 00:58:30.136508 28734 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 025dbc759c9a4ed3986536b4df6c761d in term 3.
I20260810 00:58:30.136984 28494 leader_election.cc:302] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 025dbc759c9a4ed3986536b4df6c761d, 6f8bda17b7544088b413ba1a962fdd6d; no voters: 31ff0eea6672410291c03a7df56d1cf2
I20260810 00:58:30.137326 29348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 3 FOLLOWER]: Leader election won for term 3
I20260810 00:58:30.137758 29348 raft_consensus.cc:686] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [term 3 LEADER]: Becoming Leader. State: Replica: 025dbc759c9a4ed3986536b4df6c761d, State: Running, Role: LEADER
I20260810 00:58:30.138180 29348 consensus_queue.cc:237] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: false } }
I20260810 00:58:30.140389 29356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 025dbc759c9a4ed3986536b4df6c761d. Latest consensus state: current_term: 3 leader_uuid: "025dbc759c9a4ed3986536b4df6c761d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: false } } }
I20260810 00:58:30.140749 29356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:30.141440 29358 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:58:30.144958 29358 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:58:30.145778 29358 catalog_manager.cc:1259] Loaded cluster ID: 8c10a06edd3847a3bd451ed9039cadf6
I20260810 00:58:30.145880 29358 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:58:30.147073 29358 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:58:30.147884 29358 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d: Loaded TSK: 0
I20260810 00:58:30.148736 29358 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:58:30.208599 28509 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:58:30.215658 28509 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:58:30.221657 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.220971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.0.0.1:47148)
0810 00:58:30.221351 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.221467 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260810 00:58:30.222247 28734 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [term 3 FOLLOWER]: Refusing update from remote peer 025dbc759c9a4ed3986536b4df6c761d: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260810 00:58:30.223116 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.222593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.0.0.1:47150)
0810 00:58:30.222857 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.222960 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260810 00:58:30.223376 28494 consensus_peers.cc:487] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d -> Peer 31ff0eea6672410291c03a7df56d1cf2 (127.20.81.190:33681): Couldn't send request to peer 31ff0eea6672410291c03a7df56d1cf2. Status: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 00:58:30.224164 29356 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260810 00:58:30.231094 29361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 025dbc759c9a4ed3986536b4df6c761d. Latest consensus state: current_term: 3 leader_uuid: "025dbc759c9a4ed3986536b4df6c761d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: false } } }
I20260810 00:58:30.231397 29361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:58:30.236416 29348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "025dbc759c9a4ed3986536b4df6c761d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: false } } }
I20260810 00:58:30.236744 29348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:30.238309 29361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "025dbc759c9a4ed3986536b4df6c761d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31ff0eea6672410291c03a7df56d1cf2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33681 } } peers { permanent_uuid: "025dbc759c9a4ed3986536b4df6c761d" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 46407 } attrs { promote: false } } peers { permanent_uuid: "6f8bda17b7544088b413ba1a962fdd6d" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 45483 } attrs { promote: false } } }
I20260810 00:58:30.238637 29361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f8bda17b7544088b413ba1a962fdd6d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:58:30.256825 29376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.245742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:35525 (local address 127.0.0.1:49740)
0810 00:58:30.248180 (+ 2438us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.248201 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:30.248317 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:30.249189 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:30.249196 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:30.249229 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:30.249713 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:30.249724 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.251329 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.251344 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:30.253462 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:30.253481 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.255759 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.255773 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:30.256020 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:30.256074 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 00:58:30.256187 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 00:58:30.256375 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2299,"thread_start_us":2144,"threads_started":1}
I20260810 00:58:30.257629 29378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.245834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49740 (local address 127.20.81.131:35525)
0810 00:58:30.248853 (+ 3019us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:30.248860 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:30.248883 (+ 23us) server_negotiation.cc:407] Connection header received
0810 00:58:30.248968 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:30.248976 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:30.249053 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:30.249158 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:30.249923 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.251207 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.253653 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.255465 (+ 1812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.255644 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:30.255842 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:30.257076 (+ 1234us) server_negotiation.cc:299] Negotiation successful
0810 00:58:30.257239 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2861,"thread_start_us":2786,"threads_started":1}
I20260810 00:58:30.258724 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.244866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:36539 (local address 127.0.0.1:40706)
0810 00:58:30.245417 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.245438 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:30.245565 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:30.246096 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:30.246105 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:30.246143 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:30.246591 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:30.246608 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.249198 (+ 2590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.249204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:30.250926 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:30.250938 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.257962 (+ 7024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.257971 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:30.258175 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:30.258215 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 00:58:30.258286 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 00:58:30.258466 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440}
I20260810 00:58:30.258790 29175 tablet_service.cc:1467] Processing CreateTablet for tablet 9c58a2baca3749f8ae1bc5e5f22ee09b (DEFAULT_TABLE table=test-table [id=171b1b21e139467295dd5ced7c060e8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:30.259382 29175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c58a2baca3749f8ae1bc5e5f22ee09b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:30.259547 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.245069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40706 (local address 127.20.81.130:36539)
0810 00:58:30.245717 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:30.245725 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:30.245743 (+ 18us) server_negotiation.cc:407] Connection header received
0810 00:58:30.245817 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:30.245821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:30.245890 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:30.245971 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:30.246931 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.248041 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.251117 (+ 3076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.255541 (+ 4424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.258950 (+ 3409us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:30.259048 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:30.259100 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 00:58:30.259225 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":400,"threads_started":1}
I20260810 00:58:30.260205 29017 tablet_service.cc:1467] Processing CreateTablet for tablet 9c58a2baca3749f8ae1bc5e5f22ee09b (DEFAULT_TABLE table=test-table [id=171b1b21e139467295dd5ced7c060e8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:30.260601 29017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c58a2baca3749f8ae1bc5e5f22ee09b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:30.266902 29383 tablet_bootstrap.cc:492] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 94f458e21af3447e8489ca2b0f0c2e07: Bootstrap starting.
I20260810 00:58:30.268633 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.247491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:33253 (local address 127.0.0.1:55080)
0810 00:58:30.249040 (+ 1549us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.249057 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:30.249174 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:30.262182 (+ 13008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:30.262187 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:30.262215 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:30.262570 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:30.262582 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.264114 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.264120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:30.266150 (+ 2030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:30.266163 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.267852 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.267858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:30.268042 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:30.268076 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 00:58:30.268149 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 00:58:30.268276 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1102,"thread_start_us":957,"threads_started":1}
I20260810 00:58:30.269567 29383 tablet_bootstrap.cc:654] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 94f458e21af3447e8489ca2b0f0c2e07: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:30.269688 29381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.247919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55080 (local address 127.20.81.129:33253)
0810 00:58:30.261713 (+ 13794us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:30.261720 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:30.261741 (+ 21us) server_negotiation.cc:407] Connection header received
0810 00:58:30.261816 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:30.261821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:30.261910 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:30.262728 (+ 818us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:30.262915 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.263943 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.266372 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.267603 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.269011 (+ 1408us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:30.269121 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:30.269175 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 00:58:30.269353 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13647,"thread_start_us":14094,"threads_started":1}
I20260810 00:58:30.270448 28851 tablet_service.cc:1467] Processing CreateTablet for tablet 9c58a2baca3749f8ae1bc5e5f22ee09b (DEFAULT_TABLE table=test-table [id=171b1b21e139467295dd5ced7c060e8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:30.271036 28851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c58a2baca3749f8ae1bc5e5f22ee09b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:30.271816 29382 tablet_bootstrap.cc:492] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 4b439751b0f64ffd8322d2894db470ac: Bootstrap starting.
I20260810 00:58:30.273344 29382 tablet_bootstrap.cc:654] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 4b439751b0f64ffd8322d2894db470ac: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:30.279151 29382 tablet_bootstrap.cc:492] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 4b439751b0f64ffd8322d2894db470ac: No bootstrap required, opened a new log
I20260810 00:58:30.279338 29382 ts_tablet_manager.cc:1397] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 4b439751b0f64ffd8322d2894db470ac: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260810 00:58:30.279577 29386 tablet_bootstrap.cc:492] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7: Bootstrap starting.
I20260810 00:58:30.280133 29382 raft_consensus.cc:348] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 4b439751b0f64ffd8322d2894db470ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b439751b0f64ffd8322d2894db470ac" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36539 } } peers { permanent_uuid: "94f458e21af3447e8489ca2b0f0c2e07" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35525 } } peers { permanent_uuid: "64786d2c70b344f8954cb3344937e3b7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33253 } }
I20260810 00:58:30.280447 29382 raft_consensus.cc:374] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 4b439751b0f64ffd8322d2894db470ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:30.280550 29382 raft_consensus.cc:729] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 4b439751b0f64ffd8322d2894db470ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b439751b0f64ffd8322d2894db470ac, State: Initialized, Role: FOLLOWER
I20260810 00:58:30.280802 29383 tablet_bootstrap.cc:492] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 94f458e21af3447e8489ca2b0f0c2e07: No bootstrap required, opened a new log
I20260810 00:58:30.280961 29383 ts_tablet_manager.cc:1397] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 94f458e21af3447e8489ca2b0f0c2e07: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.006s
I20260810 00:58:30.280869 29382 consensus_queue.cc:260] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 4b439751b0f64ffd8322d2894db470ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b439751b0f64ffd8322d2894db470ac" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36539 } } peers { permanent_uuid: "94f458e21af3447e8489ca2b0f0c2e07" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35525 } } peers { permanent_uuid: "64786d2c70b344f8954cb3344937e3b7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33253 } }
I20260810 00:58:30.281462 29386 tablet_bootstrap.cc:654] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:30.281819 29382 ts_tablet_manager.cc:1428] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 4b439751b0f64ffd8322d2894db470ac: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 00:58:30.281888 29383 raft_consensus.cc:348] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 94f458e21af3447e8489ca2b0f0c2e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b439751b0f64ffd8322d2894db470ac" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36539 } } peers { permanent_uuid: "94f458e21af3447e8489ca2b0f0c2e07" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35525 } } peers { permanent_uuid: "64786d2c70b344f8954cb3344937e3b7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33253 } }
I20260810 00:58:30.282110 29383 raft_consensus.cc:374] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 94f458e21af3447e8489ca2b0f0c2e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:30.282209 29383 raft_consensus.cc:729] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 94f458e21af3447e8489ca2b0f0c2e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94f458e21af3447e8489ca2b0f0c2e07, State: Initialized, Role: FOLLOWER
I20260810 00:58:30.282495 29383 consensus_queue.cc:260] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 94f458e21af3447e8489ca2b0f0c2e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b439751b0f64ffd8322d2894db470ac" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36539 } } peers { permanent_uuid: "94f458e21af3447e8489ca2b0f0c2e07" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35525 } } peers { permanent_uuid: "64786d2c70b344f8954cb3344937e3b7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33253 } }
I20260810 00:58:30.283178 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.282455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.130:41079)
0810 00:58:30.282825 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.282970 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249}
I20260810 00:58:30.286173 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.285580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.130:59395)
0810 00:58:30.285858 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.285961 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260810 00:58:30.286844 29096 heartbeater.cc:503] Master 127.20.81.189:46407 was elected leader, sending a full tablet report...
I20260810 00:58:30.287134 29383 ts_tablet_manager.cc:1428] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 94f458e21af3447e8489ca2b0f0c2e07: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260810 00:58:30.288223 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.287753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.131:34471)
0810 00:58:30.287943 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.288067 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260810 00:58:30.291723 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.291125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.131:55787)
0810 00:58:30.291458 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.291575 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231}
I20260810 00:58:30.294396 29386 tablet_bootstrap.cc:492] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7: No bootstrap required, opened a new log
I20260810 00:58:30.294683 29386 ts_tablet_manager.cc:1397] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.009s
I20260810 00:58:30.295560 29386 raft_consensus.cc:348] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b439751b0f64ffd8322d2894db470ac" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36539 } } peers { permanent_uuid: "94f458e21af3447e8489ca2b0f0c2e07" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35525 } } peers { permanent_uuid: "64786d2c70b344f8954cb3344937e3b7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33253 } }
I20260810 00:58:30.295773 29386 raft_consensus.cc:374] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:30.295895 29386 raft_consensus.cc:729] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64786d2c70b344f8954cb3344937e3b7, State: Initialized, Role: FOLLOWER
I20260810 00:58:30.296154 29386 consensus_queue.cc:260] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b439751b0f64ffd8322d2894db470ac" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36539 } } peers { permanent_uuid: "94f458e21af3447e8489ca2b0f0c2e07" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35525 } } peers { permanent_uuid: "64786d2c70b344f8954cb3344937e3b7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33253 } }
I20260810 00:58:30.297441 29386 ts_tablet_manager.cc:1428] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260810 00:58:30.298143 29345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.297486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.129:32973)
0810 00:58:30.297875 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.297969 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264}
I20260810 00:58:30.298858 28924 heartbeater.cc:503] Master 127.20.81.189:46407 was elected leader, sending a full tablet report...
I20260810 00:58:30.299247 29345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.298861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.129:51677)
0810 00:58:30.299010 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.299089 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 00:58:30.529251 29391 raft_consensus.cc:482] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:58:30.529582 29391 raft_consensus.cc:504] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b439751b0f64ffd8322d2894db470ac" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36539 } } peers { permanent_uuid: "94f458e21af3447e8489ca2b0f0c2e07" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35525 } } peers { permanent_uuid: "64786d2c70b344f8954cb3344937e3b7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33253 } }
I20260810 00:58:30.531344 29391 leader_election.cc:288] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b439751b0f64ffd8322d2894db470ac (127.20.81.130:36539), 94f458e21af3447e8489ca2b0f0c2e07 (127.20.81.131:35525)
I20260810 00:58:30.537344 29378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.531817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:58213 (local address 127.20.81.131:35525)
0810 00:58:30.532073 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:30.532077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:30.532094 (+ 17us) server_negotiation.cc:407] Connection header received
0810 00:58:30.532161 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:30.532165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:30.532222 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:30.532308 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:30.533019 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.533778 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.535645 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.536399 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.536580 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:30.536659 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:30.536947 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 00:58:30.537079 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 00:58:30.537379 29345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.531679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:35525 (local address 127.20.81.129:58213)
0810 00:58:30.531825 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.531846 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:30.532023 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:30.532369 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:30.532376 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:30.532416 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:30.532847 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:30.532863 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.533932 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.533937 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:30.535472 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:30.535484 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.536615 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.536621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:30.536784 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:30.536807 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 00:58:30.536918 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 00:58:30.537066 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 00:58:30.538242 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.531817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:43549 (local address 127.20.81.130:36539)
0810 00:58:30.532046 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:30.532053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:30.532946 (+ 893us) server_negotiation.cc:407] Connection header received
0810 00:58:30.533161 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:30.533165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:30.533221 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:30.533318 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:30.533781 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.534654 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.536477 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.537243 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.537423 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:30.537508 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:30.537850 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 00:58:30.537985 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 00:58:30.538189 29197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c58a2baca3749f8ae1bc5e5f22ee09b" candidate_uuid: "64786d2c70b344f8954cb3344937e3b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94f458e21af3447e8489ca2b0f0c2e07" is_pre_election: true
I20260810 00:58:30.538662 29197 raft_consensus.cc:2454] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 94f458e21af3447e8489ca2b0f0c2e07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64786d2c70b344f8954cb3344937e3b7 in term 0.
I20260810 00:58:30.539057 29392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.531679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:36539 (local address 127.20.81.129:43549)
0810 00:58:30.532838 (+ 1159us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.532860 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:30.533010 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:30.533331 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:30.533336 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:30.533359 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:30.533639 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:30.533646 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.534790 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.534797 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:30.536328 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:30.536340 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.537468 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.537475 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:30.537683 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:30.537706 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 00:58:30.538484 (+ 778us) client_negotiation.cc:240] Negotiation successful
0810 00:58:30.538619 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1051,"mutex_wait_us":449,"thread_start_us":396,"threads_started":1}
I20260810 00:58:30.539259 28799 leader_election.cc:302] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64786d2c70b344f8954cb3344937e3b7, 94f458e21af3447e8489ca2b0f0c2e07; no voters:
I20260810 00:58:30.539649 29391 raft_consensus.cc:2781] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:58:30.539784 29391 raft_consensus.cc:482] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:58:30.539697 29039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c58a2baca3749f8ae1bc5e5f22ee09b" candidate_uuid: "64786d2c70b344f8954cb3344937e3b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b439751b0f64ffd8322d2894db470ac" is_pre_election: true
I20260810 00:58:30.539884 29391 raft_consensus.cc:3037] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:30.539963 29039 raft_consensus.cc:2454] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 4b439751b0f64ffd8322d2894db470ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64786d2c70b344f8954cb3344937e3b7 in term 0.
I20260810 00:58:30.542160 29391 raft_consensus.cc:504] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b439751b0f64ffd8322d2894db470ac" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36539 } } peers { permanent_uuid: "94f458e21af3447e8489ca2b0f0c2e07" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35525 } } peers { permanent_uuid: "64786d2c70b344f8954cb3344937e3b7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33253 } }
I20260810 00:58:30.542759 29391 leader_election.cc:288] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7 [CANDIDATE]: Term 1 election: Requested vote from peers 4b439751b0f64ffd8322d2894db470ac (127.20.81.130:36539), 94f458e21af3447e8489ca2b0f0c2e07 (127.20.81.131:35525)
I20260810 00:58:30.543226 29039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c58a2baca3749f8ae1bc5e5f22ee09b" candidate_uuid: "64786d2c70b344f8954cb3344937e3b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b439751b0f64ffd8322d2894db470ac"
I20260810 00:58:30.543282 29197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c58a2baca3749f8ae1bc5e5f22ee09b" candidate_uuid: "64786d2c70b344f8954cb3344937e3b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94f458e21af3447e8489ca2b0f0c2e07"
I20260810 00:58:30.543458 29039 raft_consensus.cc:3037] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 4b439751b0f64ffd8322d2894db470ac [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:30.543510 29197 raft_consensus.cc:3037] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 94f458e21af3447e8489ca2b0f0c2e07 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:30.545673 29039 raft_consensus.cc:2454] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 4b439751b0f64ffd8322d2894db470ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64786d2c70b344f8954cb3344937e3b7 in term 1.
I20260810 00:58:30.545909 29197 raft_consensus.cc:2454] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 94f458e21af3447e8489ca2b0f0c2e07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64786d2c70b344f8954cb3344937e3b7 in term 1.
I20260810 00:58:30.546123 28801 leader_election.cc:302] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b439751b0f64ffd8322d2894db470ac, 64786d2c70b344f8954cb3344937e3b7; no voters:
I20260810 00:58:30.546473 29391 raft_consensus.cc:2781] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:58:30.546929 29391 raft_consensus.cc:686] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7 [term 1 LEADER]: Becoming Leader. State: Replica: 64786d2c70b344f8954cb3344937e3b7, State: Running, Role: LEADER
I20260810 00:58:30.547379 29391 consensus_queue.cc:237] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b439751b0f64ffd8322d2894db470ac" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36539 } } peers { permanent_uuid: "94f458e21af3447e8489ca2b0f0c2e07" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35525 } } peers { permanent_uuid: "64786d2c70b344f8954cb3344937e3b7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33253 } }
I20260810 00:58:30.547960 29392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.547523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.129:35349)
0810 00:58:30.547692 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.547782 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 00:58:30.551414 28508 catalog_manager.cc:5495] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64786d2c70b344f8954cb3344937e3b7 (127.20.81.129). New cstate: current_term: 1 leader_uuid: "64786d2c70b344f8954cb3344937e3b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b439751b0f64ffd8322d2894db470ac" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94f458e21af3447e8489ca2b0f0c2e07" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64786d2c70b344f8954cb3344937e3b7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33253 } health_report { overall_health: HEALTHY } } }
I20260810 00:58:30.633788 29381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.628586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55096 (local address 127.20.81.129:33253)
0810 00:58:30.628774 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:30.628778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:30.628795 (+ 17us) server_negotiation.cc:407] Connection header received
0810 00:58:30.628883 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:30.628887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:30.628962 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:30.629112 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:58:30.629547 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.630228 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.631578 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.632074 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.632167 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:30.632588 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:58:30.633196 (+ 608us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:58:30.633321 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:30.633520 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 00:58:30.633584 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 00:58:30.638104 29039 raft_consensus.cc:1261] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 4b439751b0f64ffd8322d2894db470ac [term 1 FOLLOWER]: Refusing update from remote peer 64786d2c70b344f8954cb3344937e3b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:58:30.638512 29197 raft_consensus.cc:1261] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 94f458e21af3447e8489ca2b0f0c2e07 [term 1 FOLLOWER]: Refusing update from remote peer 64786d2c70b344f8954cb3344937e3b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:58:30.638939 29394 consensus_queue.cc:1035] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b439751b0f64ffd8322d2894db470ac" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:58:30.639345 29391 consensus_queue.cc:1035] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94f458e21af3447e8489ca2b0f0c2e07" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:58:30.641369 29250 heartbeater.cc:503] Master 127.20.81.189:46407 was elected leader, sending a full tablet report...
I20260810 00:58:30.642453 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.641756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.131:33381)
0810 00:58:30.642072 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.642193 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260810 00:58:30.642642 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.641980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.130:48935)
0810 00:58:30.642294 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.642430 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
I20260810 00:58:30.649885 29392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.648766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.129:46653)
0810 00:58:30.649584 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.649716 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":708}
I20260810 00:58:30.651451 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.650918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.131:52779)
0810 00:58:30.651152 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.651258 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260810 00:58:30.661128 28508 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:58:30.661437 28508 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:58:30.665236 29392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.664721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.129:59357)
0810 00:58:30.664939 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.665056 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260810 00:58:30.666906 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.666291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.20.81.130:48683)
0810 00:58:30.666558 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.666654 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260810 00:58:30.670954 28508 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 025dbc759c9a4ed3986536b4df6c761d: Sending DeleteTablet for 3 replicas of tablet 9c58a2baca3749f8ae1bc5e5f22ee09b
I20260810 00:58:30.672159 29175 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c58a2baca3749f8ae1bc5e5f22ee09b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:30 UTC) from {username='oryx', principal='oryx/127.20.81.189@KRBTEST.COM'} at 127.0.0.1:49740
I20260810 00:58:30.672372 28851 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c58a2baca3749f8ae1bc5e5f22ee09b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:30 UTC) from {username='oryx', principal='oryx/127.20.81.189@KRBTEST.COM'} at 127.0.0.1:55080
I20260810 00:58:30.673529 29016 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c58a2baca3749f8ae1bc5e5f22ee09b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:30 UTC) from {username='oryx', principal='oryx/127.20.81.189@KRBTEST.COM'} at 127.0.0.1:40706
I20260810 00:58:30.673714 29017 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c58a2baca3749f8ae1bc5e5f22ee09b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:30 UTC) from {username='oryx', principal='oryx/127.20.81.189@KRBTEST.COM'} at 127.0.0.1:40706
I20260810 00:58:30.674248 29411 tablet_replica.cc:330] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 94f458e21af3447e8489ca2b0f0c2e07: stopping tablet replica
I20260810 00:58:30.674476 29413 tablet_replica.cc:330] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 4b439751b0f64ffd8322d2894db470ac: stopping tablet replica
I20260810 00:58:30.674522 29411 raft_consensus.cc:2229] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 94f458e21af3447e8489ca2b0f0c2e07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:58:30.674659 29413 raft_consensus.cc:2229] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 4b439751b0f64ffd8322d2894db470ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:58:30.674860 29411 raft_consensus.cc:2258] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 94f458e21af3447e8489ca2b0f0c2e07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:30.674923 29413 raft_consensus.cc:2258] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 4b439751b0f64ffd8322d2894db470ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:30.676368 29411 ts_tablet_manager.cc:1905] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 94f458e21af3447e8489ca2b0f0c2e07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:30.677944 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 28688
I20260810 00:58:30.679107 29413 ts_tablet_manager.cc:1905] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 4b439751b0f64ffd8322d2894db470ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:30.683157 29411 ts_tablet_manager.cc:1918] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 94f458e21af3447e8489ca2b0f0c2e07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:58:30.683292 29411 log.cc:1196] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 94f458e21af3447e8489ca2b0f0c2e07: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/9c58a2baca3749f8ae1bc5e5f22ee09b
I20260810 00:58:30.683746 29411 ts_tablet_manager.cc:1939] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 94f458e21af3447e8489ca2b0f0c2e07: Deleting consensus metadata
I20260810 00:58:30.684959 28493 catalog_manager.cc:4841] TS 94f458e21af3447e8489ca2b0f0c2e07 (127.20.81.131:35525): tablet 9c58a2baca3749f8ae1bc5e5f22ee09b (table test-table [id=171b1b21e139467295dd5ced7c060e8e]) successfully deleted
I20260810 00:58:30.685753 29413 ts_tablet_manager.cc:1918] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 4b439751b0f64ffd8322d2894db470ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:58:30.685896 29413 log.cc:1196] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 4b439751b0f64ffd8322d2894db470ac: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/9c58a2baca3749f8ae1bc5e5f22ee09b
I20260810 00:58:30.686434 29413 ts_tablet_manager.cc:1939] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 4b439751b0f64ffd8322d2894db470ac: Deleting consensus metadata
I20260810 00:58:30.687641 28494 catalog_manager.cc:4841] TS 4b439751b0f64ffd8322d2894db470ac (127.20.81.130:36539): tablet 9c58a2baca3749f8ae1bc5e5f22ee09b (table test-table [id=171b1b21e139467295dd5ced7c060e8e]) successfully deleted
W20260810 00:58:30.688637 28494 catalog_manager.cc:4808] TS 4b439751b0f64ffd8322d2894db470ac (127.20.81.130:36539): delete failed for tablet 9c58a2baca3749f8ae1bc5e5f22ee09b because the tablet was not found. No further retry: Not found: Tablet not found: 9c58a2baca3749f8ae1bc5e5f22ee09b
I20260810 00:58:30.691008 29412 tablet_replica.cc:330] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7: stopping tablet replica
I20260810 00:58:30.691179 29412 raft_consensus.cc:2229] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:58:30.691710 29412 raft_consensus.cc:2258] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:30.693461 29412 ts_tablet_manager.cc:1905] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:30.701107 29412 ts_tablet_manager.cc:1918] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:58:30.701329 29412 log.cc:1196] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323445691443-20806-0/minicluster-data/ts-0/wal/wals/9c58a2baca3749f8ae1bc5e5f22ee09b
I20260810 00:58:30.701635 29381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.678707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55112 (local address 127.20.81.129:33253)
0810 00:58:30.678887 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:30.678893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:30.685755 (+ 6862us) server_negotiation.cc:407] Connection header received
0810 00:58:30.686080 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:30.686088 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:30.686217 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:30.686305 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:30.688372 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.689439 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.690843 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.691596 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.691738 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:30.695091 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:30.695107 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:30.695121 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:30.695151 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:30.698499 (+ 3348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:30.699331 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:30.699350 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:30.699353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:30.699530 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:30.699919 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:30.699924 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:30.699927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:30.700278 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:30.700818 (+ 540us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:30.701000 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 00:58:30.701225 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 00:58:30.701879 29412 ts_tablet_manager.cc:1939] T 9c58a2baca3749f8ae1bc5e5f22ee09b P 64786d2c70b344f8954cb3344937e3b7: Deleting consensus metadata
I20260810 00:58:30.702354 28916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:30.710637 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.710137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.0.0.1:47166)
0810 00:58:30.710328 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.710448 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 00:58:30.711710 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.711275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:33681 (local address 127.0.0.1:47182)
0810 00:58:30.711444 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:30.711552 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:33681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 00:58:30.821442 28492 catalog_manager.cc:4841] TS 64786d2c70b344f8954cb3344937e3b7 (127.20.81.129:33253): tablet 9c58a2baca3749f8ae1bc5e5f22ee09b (table test-table [id=171b1b21e139467295dd5ced7c060e8e]) successfully deleted
I20260810 00:58:30.821868 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 28688
I20260810 00:58:30.845324 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 28935
I20260810 00:58:30.859524 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.846127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40714 (local address 127.20.81.130:36539)
0810 00:58:30.846336 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:30.846341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:30.846360 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:58:30.846601 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:30.846605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:30.846671 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:30.846756 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:30.848075 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.848871 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.850138 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.850777 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.850878 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:30.853512 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:30.853528 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:30.853544 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:30.853582 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:30.856813 (+ 3231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:30.857509 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:30.857522 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:30.857524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:30.857661 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:30.857975 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:30.857978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:30.857980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:30.858377 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:30.858787 (+ 410us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:30.858972 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 00:58:30.859146 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 00:58:30.860411 29089 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:30.954802 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 28935
I20260810 00:58:30.976869 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 29107
I20260810 00:58:30.990633 29378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:30.977667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49754 (local address 127.20.81.131:35525)
0810 00:58:30.977864 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:30.977869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:30.977906 (+ 37us) server_negotiation.cc:407] Connection header received
0810 00:58:30.978100 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:30.978104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:30.978167 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:30.978253 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:30.979718 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.980478 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.981681 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:30.982073 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:30.982164 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:30.984772 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:30.984788 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:30.984804 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:30.984841 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:30.987902 (+ 3061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:30.988639 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:30.988646 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:30.988648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:30.988769 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:30.989122 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:30.989125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:30.989127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:30.989496 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:30.989717 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:30.990042 (+ 325us) server_negotiation.cc:299] Negotiation successful
0810 00:58:30.990221 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 00:58:30.991323 29243 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:31.114876 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 29107
I20260810 00:58:31.141393 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 28356
I20260810 00:58:31.142591 28543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:31.270151 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 28356
I20260810 00:58:31.296430 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 28465
I20260810 00:58:31.297612 28755 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:31.419209 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 28465
2026-08-10T00:58:31Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (12146 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:58:31 dist-test-slave-gp6n krb5kdc[29424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:58:31 dist-test-slave-gp6n krb5kdc[29424](info): set up 2 sockets
Aug 10 00:58:31 dist-test-slave-gp6n krb5kdc[29424](info): commencing operation
krb5kdc: starting...
W20260810 00:58:32.635054 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.007s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 00:58:32 dist-test-slave-gp6n krb5kdc[29424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:58:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:58:32Z Disabled control of system clock
WARNING: no policy specified for oryx/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.190.keytab.
Entry for principal oryx/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.190.keytab.
I20260810 00:58:32.891765 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:38311
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.190.keytab
--principal=oryx/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:33719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:38311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:33.229295 29439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:33.229590 29439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:33.229765 29439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:33.236968 29439 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:58:33.237075 29439 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 00:58:33.237164 29439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:33.237280 29439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:33.237375 29439 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:58:33.237494 29439 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:58:33.247408 29439 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:33719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:38311
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.190.keytab
--principal=oryx/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:38311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:33.248737 29439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:33.250128 29439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:33.262759 29444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:33.262714 29445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:33.262967 29447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:33.263631 29439 server_base.cc:1034] running on GCE node
I20260810 00:58:33.264432 29439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:33.265839 29439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:33.267086 29439 hybrid_clock.cc:648] HybridClock initialized: now 1786323513267015 us; error 51 us; skew 500 ppm
I20260810 00:58:33.267431 29439 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:33 dist-test-slave-gp6n krb5kdc[29424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323513, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:33.270732 29439 init.cc:377] Logged in from keytab as oryx/127.20.81.190@KRBTEST.COM (short username oryx)
I20260810 00:58:33.272169 29439 webserver.cc:460] Webserver started at http://127.20.81.190:35377/ using document root <none> and password file <none>
I20260810 00:58:33.272886 29439 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:33.273015 29439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:33.273344 29439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:33.276382 29439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "5dad884925fc477b833f1b2a578b46e4"
format_stamp: "Formatted at 2026-08-10 00:58:33 on dist-test-slave-gp6n"
server_key: "b23aad67df47bbe0fddcffe535d89245"
server_key_iv: "dd9fb0a4b4382ec03184b1d2d5641af1"
server_key_version: "encryptionkey@0"
I20260810 00:58:33.277205 29439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "5dad884925fc477b833f1b2a578b46e4"
format_stamp: "Formatted at 2026-08-10 00:58:33 on dist-test-slave-gp6n"
server_key: "b23aad67df47bbe0fddcffe535d89245"
server_key_iv: "dd9fb0a4b4382ec03184b1d2d5641af1"
server_key_version: "encryptionkey@0"
I20260810 00:58:33.282238 29439 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260810 00:58:33.285732 29454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:33.287114 29439 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:58:33.287310 29439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "5dad884925fc477b833f1b2a578b46e4"
format_stamp: "Formatted at 2026-08-10 00:58:33 on dist-test-slave-gp6n"
server_key: "b23aad67df47bbe0fddcffe535d89245"
server_key_iv: "dd9fb0a4b4382ec03184b1d2d5641af1"
server_key_version: "encryptionkey@0"
I20260810 00:58:33.287555 29439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:33.306140 29439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:33.310722 29439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:33.311017 29439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:33.334234 29439 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:38311
I20260810 00:58:33.334254 29528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:38311 every 8 connection(s)
I20260810 00:58:33.335790 29439 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:58:33.339644 29529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:33.345863 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 29439
I20260810 00:58:33.346119 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 00:58:33.346822 20806 external_mini_cluster.cc:1428] Setting key 9810874df56d91cad7f6d5cf1ff2b86f
I20260810 00:58:33.350344 29529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4: Bootstrap starting.
I20260810 00:58:33.353961 29529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:33.355199 29529 log.cc:824] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:33.358300 29529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4: No bootstrap required, opened a new log
Aug 10 00:58:33 dist-test-slave-gp6n krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM
I20260810 00:58:33.367362 29529 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dad884925fc477b833f1b2a578b46e4" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38311 } }
I20260810 00:58:33.367753 29529 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:33.367882 29529 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dad884925fc477b833f1b2a578b46e4, State: Initialized, Role: FOLLOWER
I20260810 00:58:33.368536 29529 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dad884925fc477b833f1b2a578b46e4" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38311 } }
I20260810 00:58:33.368790 29529 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:58:33.368952 29529 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:58:33.369166 29529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:33.371160 29534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:33.348831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58154 (local address 127.20.81.190:38311)
0810 00:58:33.349688 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:33.349700 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:33.349741 (+ 41us) server_negotiation.cc:407] Connection header received
0810 00:58:33.351606 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:33.351628 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:33.352042 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:33.352354 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:33.354298 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:33.355916 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:33.357624 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:33.358286 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:33.358497 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:33.363143 (+ 4646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:33.363151 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:33.363166 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:33.363211 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:33.366306 (+ 3095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:33.367022 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:33.367030 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:33.367036 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:33.367199 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:33.367658 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:33.367664 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:33.367668 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:33.368343 (+ 675us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:33.368665 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:33.369229 (+ 564us) server_negotiation.cc:299] Negotiation successful
0810 00:58:33.369725 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":430,"threads_started":1}
I20260810 00:58:33.372243 29529 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dad884925fc477b833f1b2a578b46e4" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38311 } }
I20260810 00:58:33.372818 29529 leader_election.cc:302] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5dad884925fc477b833f1b2a578b46e4; no voters:
I20260810 00:58:33.373489 29529 leader_election.cc:288] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:58:33.373623 29536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:58:33.375196 29536 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4 [term 1 LEADER]: Becoming Leader. State: Replica: 5dad884925fc477b833f1b2a578b46e4, State: Running, Role: LEADER
I20260810 00:58:33.375746 29536 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dad884925fc477b833f1b2a578b46e4" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38311 } }
I20260810 00:58:33.377946 29537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5dad884925fc477b833f1b2a578b46e4. Latest consensus state: current_term: 1 leader_uuid: "5dad884925fc477b833f1b2a578b46e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dad884925fc477b833f1b2a578b46e4" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38311 } } }
I20260810 00:58:33.379194 29537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:33.379637 29529 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:58:33.379741 29538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5dad884925fc477b833f1b2a578b46e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dad884925fc477b833f1b2a578b46e4" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38311 } } }
I20260810 00:58:33.380106 29538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:33.380817 29540 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:58:33.390275 29540 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:58:33.400306 29540 catalog_manager.cc:1347] Generated new cluster ID: 49b6eff4605e466796b70fd04f608c42
I20260810 00:58:33.400470 29540 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:58:33.416286 29540 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:58:33.418324 29540 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:58:33.431100 29540 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4: Generated new TSK 0
I20260810 00:58:33.432231 29540 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.20.81.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.81.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.81.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.129.keytab.
Entry for principal oryx/127.20.81.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.129.keytab.
I20260810 00:58:33.496165 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:38311
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.129.keytab
--principal=oryx/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:33719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:33.840308 29557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:33.840590 29557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:33.840700 29557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:33.847729 29557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:33.847863 29557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:33.847951 29557 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 00:58:33.848033 29557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 00:58:33.857591 29557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:33719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.129.keytab
--principal=oryx/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.81.190:38311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:33.858912 29557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:33.860286 29557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:33.875996 29563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:33.876320 29565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:33.876506 29562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:33.880136 29557 server_base.cc:1034] running on GCE node
I20260810 00:58:33.880995 29557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:33.882053 29557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:33.883304 29557 hybrid_clock.cc:648] HybridClock initialized: now 1786323513883218 us; error 61 us; skew 500 ppm
I20260810 00:58:33.883708 29557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:33 dist-test-slave-gp6n krb5kdc[29424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323513, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:33.887602 29557 init.cc:377] Logged in from keytab as oryx/127.20.81.129@KRBTEST.COM (short username oryx)
I20260810 00:58:33.891431 29557 webserver.cc:460] Webserver started at http://127.20.81.129:40151/ using document root <none> and password file <none>
I20260810 00:58:33.892459 29557 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:33.892590 29557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:33.893010 29557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:33.897297 29557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "0ac5028bde95499e912b1bb99ba8a7d9"
format_stamp: "Formatted at 2026-08-10 00:58:33 on dist-test-slave-gp6n"
server_key: "f8194845cd1f939c92026fdf7a73e95b"
server_key_iv: "bc8d6a124d6a0165772b1426f8338f9c"
server_key_version: "encryptionkey@0"
I20260810 00:58:33.898447 29557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "0ac5028bde95499e912b1bb99ba8a7d9"
format_stamp: "Formatted at 2026-08-10 00:58:33 on dist-test-slave-gp6n"
server_key: "f8194845cd1f939c92026fdf7a73e95b"
server_key_iv: "bc8d6a124d6a0165772b1426f8338f9c"
server_key_version: "encryptionkey@0"
I20260810 00:58:33.905525 29557 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.005s
I20260810 00:58:33.909564 29573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:33.911110 29557 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:58:33.911283 29557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "0ac5028bde95499e912b1bb99ba8a7d9"
format_stamp: "Formatted at 2026-08-10 00:58:33 on dist-test-slave-gp6n"
server_key: "f8194845cd1f939c92026fdf7a73e95b"
server_key_iv: "bc8d6a124d6a0165772b1426f8338f9c"
server_key_version: "encryptionkey@0"
I20260810 00:58:33.911527 29557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:33.925405 29557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:33.929299 29557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:33.929581 29557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:33.932515 29557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:33.932641 29557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:33.932785 29557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:33.932849 29557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 00:58:33 dist-test-slave-gp6n krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323513, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM
I20260810 00:58:33.957676 29582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:33.932820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:60085 (local address 127.20.81.190:38311)
0810 00:58:33.933302 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:33.933309 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:33.934663 (+ 1354us) server_negotiation.cc:407] Connection header received
0810 00:58:33.937729 (+ 3066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:33.937738 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:33.937852 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:33.937973 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:33.940618 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:33.941376 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:33.944967 (+ 3591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:33.945402 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:33.945512 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:33.951210 (+ 5698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:33.951215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:33.951219 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:33.951255 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:33.953988 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:33.954830 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:33.954838 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:33.954846 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:33.954978 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:33.955586 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:33.955591 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:33.955594 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:33.955889 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:33.956076 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:33.957239 (+ 1163us) server_negotiation.cc:299] Negotiation successful
0810 00:58:33.957426 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":256,"threads_started":1}
I20260810 00:58:33.959792 29584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:33.933495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38311 (local address 127.20.81.129:60085)
0810 00:58:33.934480 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:33.934535 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:33.937332 (+ 2797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:33.938292 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:33.938304 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:33.939314 (+ 1010us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:33.940379 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:33.940408 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:33.941564 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:33.941576 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:33.944766 (+ 3190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:33.944782 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:33.945665 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:33.945675 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:33.946065 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:33.946993 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:33.947034 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:33.950928 (+ 3894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:33.954138 (+ 3210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:33.954147 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:33.954163 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:33.954689 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:33.955103 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:33.955111 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:33.955114 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:33.955408 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:33.956070 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:33.956081 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:33.956697 (+ 616us) client_negotiation.cc:769] Sending connection context
0810 00:58:33.957134 (+ 437us) client_negotiation.cc:240] Negotiation successful
0810 00:58:33.957870 (+ 736us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":385,"threads_started":1}
I20260810 00:58:33.982896 29557 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:45663
I20260810 00:58:33.982944 29708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:45663 every 8 connection(s)
I20260810 00:58:33.984480 29557 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 00:58:33.986888 29709 heartbeater.cc:347] Connected to a master server at 127.20.81.190:38311
I20260810 00:58:33.986939 29669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:33.968424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38311 (local address 127.20.81.129:51639)
0810 00:58:33.969262 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:33.969295 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:33.969420 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:33.969964 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:33.969973 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:33.970887 (+ 914us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:33.971549 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:33.971567 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:33.973244 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:33.973249 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:33.975270 (+ 2021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:33.975290 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:33.976777 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:33.976786 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:33.977020 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:33.977810 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:33.977848 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:33.979326 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:33.983508 (+ 4182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:33.983518 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:33.983522 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:33.984057 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:33.984496 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:33.984503 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:33.984507 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:33.984661 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:33.985464 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:33.985472 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:33.985755 (+ 283us) client_negotiation.cc:769] Sending connection context
0810 00:58:33.985952 (+ 197us) client_negotiation.cc:240] Negotiation successful
0810 00:58:33.986339 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":545,"threads_started":1}
I20260810 00:58:33.987165 29709 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:33.987542 29582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:33.968349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:51639 (local address 127.20.81.190:38311)
0810 00:58:33.968582 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:33.968591 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:33.969399 (+ 808us) server_negotiation.cc:407] Connection header received
0810 00:58:33.969524 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:33.969532 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:33.969630 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:33.969738 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:33.971777 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:33.973056 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:33.975503 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:33.976297 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:33.976433 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:33.979519 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:33.979524 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:33.979527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:33.979559 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:33.983388 (+ 3829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:33.984225 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:33.984229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:33.984231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:33.984368 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:33.984832 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:33.984837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:33.984839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:33.985287 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:33.985481 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:33.986999 (+ 1518us) server_negotiation.cc:299] Negotiation successful
0810 00:58:33.987259 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 00:58:33.987970 29709 heartbeater.cc:511] Master 127.20.81.190:38311 requested a full tablet report, sending...
I20260810 00:58:33.990216 29476 ts_manager.cc:194] Registered new tserver with Master: 0ac5028bde95499e912b1bb99ba8a7d9 (127.20.81.129:45663)
I20260810 00:58:33.991923 29476 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:60085
I20260810 00:58:33.993445 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 29557
I20260810 00:58:33.993676 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 00:58:33.994563 20806 external_mini_cluster.cc:1428] Setting key d233626fe735b9b6b82845f55059c371
WARNING: no policy specified for oryx/127.20.81.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.81.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.81.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.130.keytab.
Entry for principal oryx/127.20.81.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.130.keytab.
I20260810 00:58:34.054919 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:38311
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.130.keytab
--principal=oryx/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:33719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:34.395843 29714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:34.396126 29714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:34.396226 29714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:34.403209 29714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:34.403319 29714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:34.403402 29714 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 00:58:34.403486 29714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 00:58:34.413615 29714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:33719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.130.keytab
--principal=oryx/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.81.190:38311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:34.414994 29714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:34.416481 29714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:34.429587 29723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:34.429672 29720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:34.430158 29719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:34.430953 29714 server_base.cc:1034] running on GCE node
I20260810 00:58:34.431659 29714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:34.432555 29714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:34.433772 29714 hybrid_clock.cc:648] HybridClock initialized: now 1786323514433690 us; error 54 us; skew 500 ppm
I20260810 00:58:34.434037 29714 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:34 dist-test-slave-gp6n krb5kdc[29424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323514, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:34.437373 29714 init.cc:377] Logged in from keytab as oryx/127.20.81.130@KRBTEST.COM (short username oryx)
I20260810 00:58:34.438838 29714 webserver.cc:460] Webserver started at http://127.20.81.130:46877/ using document root <none> and password file <none>
I20260810 00:58:34.439497 29714 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:34.439584 29714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:34.439854 29714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:34.442889 29714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "82198c3537184383a48c9538ce62b11a"
format_stamp: "Formatted at 2026-08-10 00:58:34 on dist-test-slave-gp6n"
server_key: "53baad62ddf02f44318cee3bebba5a10"
server_key_iv: "c28ff4d88a00e6c3ffee89c24d3f6dc5"
server_key_version: "encryptionkey@0"
I20260810 00:58:34.443641 29714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "82198c3537184383a48c9538ce62b11a"
format_stamp: "Formatted at 2026-08-10 00:58:34 on dist-test-slave-gp6n"
server_key: "53baad62ddf02f44318cee3bebba5a10"
server_key_iv: "c28ff4d88a00e6c3ffee89c24d3f6dc5"
server_key_version: "encryptionkey@0"
I20260810 00:58:34.448791 29714 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 00:58:34.452631 29732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:34.454088 29714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 00:58:34.454291 29714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "82198c3537184383a48c9538ce62b11a"
format_stamp: "Formatted at 2026-08-10 00:58:34 on dist-test-slave-gp6n"
server_key: "53baad62ddf02f44318cee3bebba5a10"
server_key_iv: "c28ff4d88a00e6c3ffee89c24d3f6dc5"
server_key_version: "encryptionkey@0"
I20260810 00:58:34.454620 29714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:34.469940 29714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:34.474594 29714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:34.474849 29714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:34.477705 29714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:34.477818 29714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:34.478000 29714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:34.478084 29714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 00:58:34 dist-test-slave-gp6n krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323514, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM
I20260810 00:58:34.503070 29582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:34.478288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:50801 (local address 127.20.81.190:38311)
0810 00:58:34.478618 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:34.478627 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:34.480098 (+ 1471us) server_negotiation.cc:407] Connection header received
0810 00:58:34.482383 (+ 2285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:34.482389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:34.482526 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:34.482644 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:34.484853 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:34.486318 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:34.488971 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:34.489622 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:34.489755 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:34.495567 (+ 5812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:34.495573 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:34.495578 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:34.495622 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:34.499791 (+ 4169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:34.500532 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:34.500542 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:34.500549 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:34.500761 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:34.501247 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:34.501260 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:34.501262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:34.501556 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:34.501724 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:34.502462 (+ 738us) server_negotiation.cc:299] Negotiation successful
0810 00:58:34.502689 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 00:58:34.504467 29741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:34.479016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38311 (local address 127.20.81.130:50801)
0810 00:58:34.479909 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:34.479961 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:34.482098 (+ 2137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:34.482884 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:34.482892 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:34.483699 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:34.484540 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:34.484564 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:34.486534 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:34.486540 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:34.488807 (+ 2267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:34.488824 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:34.489881 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:34.489889 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:34.490111 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:34.491188 (+ 1077us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:34.491211 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:34.495282 (+ 4071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:34.499964 (+ 4682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:34.499972 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:34.499990 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:34.500396 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:34.500890 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:34.500893 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:34.500895 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:34.501123 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:34.501706 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:34.501713 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:34.502079 (+ 366us) client_negotiation.cc:769] Sending connection context
0810 00:58:34.502344 (+ 265us) client_negotiation.cc:240] Negotiation successful
0810 00:58:34.502950 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":394,"threads_started":1}
I20260810 00:58:34.525055 29582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:34.510524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:47983 (local address 127.20.81.190:38311)
0810 00:58:34.510756 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:34.510764 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:34.511494 (+ 730us) server_negotiation.cc:407] Connection header received
0810 00:58:34.511704 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:34.511709 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:34.511795 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:34.511893 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:34.513179 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:34.514078 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:34.516114 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:34.516724 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:34.516852 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:34.519295 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:34.519301 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:34.519306 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:34.519344 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:34.522207 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:34.523108 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:34.523114 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:34.523118 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:34.523252 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:34.523629 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:34.523634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:34.523643 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:34.523953 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:34.524112 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:34.524498 (+ 386us) server_negotiation.cc:299] Negotiation successful
0810 00:58:34.524705 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 00:58:34.525190 29814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:34.510609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38311 (local address 127.20.81.130:47983)
0810 00:58:34.511370 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:34.511391 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:34.511565 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:34.511916 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:34.511926 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:34.512509 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:34.512974 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:34.512995 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:34.514328 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:34.514333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:34.515953 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:34.515965 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:34.516935 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:34.516941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:34.517103 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:34.517806 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:34.517837 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:34.519095 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:34.522340 (+ 3245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:34.522350 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:34.522354 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:34.522973 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:34.523365 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:34.523369 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:34.523373 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:34.523512 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:34.524074 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:34.524079 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:34.524256 (+ 177us) client_negotiation.cc:769] Sending connection context
0810 00:58:34.524389 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 00:58:34.524705 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":398,"threads_started":1}
I20260810 00:58:34.536520 29714 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:45399
I20260810 00:58:34.536536 29864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:45399 every 8 connection(s)
I20260810 00:58:34.538280 29714 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 00:58:34.540949 29865 heartbeater.cc:347] Connected to a master server at 127.20.81.190:38311
I20260810 00:58:34.541196 29865 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:34.542023 29865 heartbeater.cc:511] Master 127.20.81.190:38311 requested a full tablet report, sending...
I20260810 00:58:34.542260 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 29714
I20260810 00:58:34.542536 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 00:58:34.543107 20806 external_mini_cluster.cc:1428] Setting key 79908748f7da056e1ba6c411c190703a
I20260810 00:58:34.543780 29476 ts_manager.cc:194] Registered new tserver with Master: 82198c3537184383a48c9538ce62b11a (127.20.81.130:45399)
I20260810 00:58:34.545236 29476 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:50801
WARNING: no policy specified for oryx/127.20.81.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.81.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.81.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.131.keytab.
Entry for principal oryx/127.20.81.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.131.keytab.
I20260810 00:58:34.604122 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:38311
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.131.keytab
--principal=oryx/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:33719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:34.941987 29869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:34.942317 29869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:34.942525 29869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:34.949826 29869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:34.949985 29869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:34.950143 29869 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 00:58:34.950287 29869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 00:58:34.960570 29869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:33719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.131.keytab
--principal=oryx/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.81.190:38311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:34.961890 29869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:34.963353 29869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:34.976354 29874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:34.976421 29875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:34.976372 29877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:34.977600 29869 server_base.cc:1034] running on GCE node
I20260810 00:58:34.978191 29869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:34.979157 29869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:34.980423 29869 hybrid_clock.cc:648] HybridClock initialized: now 1786323514980357 us; error 40 us; skew 500 ppm
I20260810 00:58:34.980746 29869 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:34 dist-test-slave-gp6n krb5kdc[29424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323514, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:34.984560 29869 init.cc:377] Logged in from keytab as oryx/127.20.81.131@KRBTEST.COM (short username oryx)
I20260810 00:58:34.988739 29869 webserver.cc:460] Webserver started at http://127.20.81.131:46589/ using document root <none> and password file <none>
I20260810 00:58:34.989475 29869 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:34.989622 29869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:34.989981 29869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:34.993180 29869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "49a07267d0814d14ba4acdf42c845584"
format_stamp: "Formatted at 2026-08-10 00:58:34 on dist-test-slave-gp6n"
server_key: "2e6067b29c03af61e9874250366d25b9"
server_key_iv: "9f807962b8554dfda318edc8347c9b50"
server_key_version: "encryptionkey@0"
I20260810 00:58:34.993970 29869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "49a07267d0814d14ba4acdf42c845584"
format_stamp: "Formatted at 2026-08-10 00:58:34 on dist-test-slave-gp6n"
server_key: "2e6067b29c03af61e9874250366d25b9"
server_key_iv: "9f807962b8554dfda318edc8347c9b50"
server_key_version: "encryptionkey@0"
I20260810 00:58:34.995316 29709 heartbeater.cc:503] Master 127.20.81.190:38311 was elected leader, sending a full tablet report...
I20260810 00:58:34.999897 29869 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 00:58:35.003355 29885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:35.004714 29869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260810 00:58:35.004904 29869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "49a07267d0814d14ba4acdf42c845584"
format_stamp: "Formatted at 2026-08-10 00:58:34 on dist-test-slave-gp6n"
server_key: "2e6067b29c03af61e9874250366d25b9"
server_key_iv: "9f807962b8554dfda318edc8347c9b50"
server_key_version: "encryptionkey@0"
I20260810 00:58:35.005137 29869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:35.022231 29869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:35.026829 29869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:35.027110 29869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:35.029897 29869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:35.030133 29869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:35.030354 29869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:35.030470 29869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 10 00:58:35 dist-test-slave-gp6n krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323514, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM
I20260810 00:58:35.054620 29894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:35.030383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:59215 (local address 127.20.81.190:38311)
0810 00:58:35.030931 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:35.030937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:35.032137 (+ 1200us) server_negotiation.cc:407] Connection header received
0810 00:58:35.034247 (+ 2110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:35.034254 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:35.034390 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:35.034562 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:35.036315 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:35.037840 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:35.040727 (+ 2887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:35.041362 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:35.041508 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:35.046497 (+ 4989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:35.046507 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:35.046514 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:35.046572 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:35.050201 (+ 3629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:35.051220 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:35.051227 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:35.051231 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:35.051375 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:35.051987 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:35.051992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:35.051996 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:35.052444 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:35.052623 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:35.053656 (+ 1033us) server_negotiation.cc:299] Negotiation successful
0810 00:58:35.053993 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":258,"threads_started":1}
I20260810 00:58:35.055562 29897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:35.031076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38311 (local address 127.20.81.131:59215)
0810 00:58:35.031955 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:35.032006 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:35.033876 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:35.034823 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:35.034837 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:35.035521 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:35.036107 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:35.036125 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:35.038013 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:35.038016 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:35.040571 (+ 2555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:35.040588 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:35.041627 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:35.041638 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:35.041883 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:35.042805 (+ 922us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:35.042848 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:35.046156 (+ 3308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:35.050339 (+ 4183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:35.050348 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:35.050365 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:35.051074 (+ 709us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:35.051499 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:35.051504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:35.051510 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:35.051834 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:35.052586 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:35.052602 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:35.053165 (+ 563us) client_negotiation.cc:769] Sending connection context
0810 00:58:35.053493 (+ 328us) client_negotiation.cc:240] Negotiation successful
0810 00:58:35.054085 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":364,"threads_started":1}
I20260810 00:58:35.079447 29869 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:33811
I20260810 00:58:35.079471 30025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:33811 every 8 connection(s)
I20260810 00:58:35.080359 29894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:35.063473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:43989 (local address 127.20.81.190:38311)
0810 00:58:35.063632 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:35.063636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:35.064583 (+ 947us) server_negotiation.cc:407] Connection header received
0810 00:58:35.064797 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:35.064807 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:35.064888 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:35.064995 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:35.065909 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:35.067150 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:35.069650 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:35.070209 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:35.070354 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:35.073695 (+ 3341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:35.073705 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:35.073712 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:35.073760 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:35.077081 (+ 3321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:35.077888 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:35.077893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:35.077895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:35.078029 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:35.078613 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:35.078621 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:35.078626 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:35.079043 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:35.079203 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:35.079893 (+ 690us) server_negotiation.cc:299] Negotiation successful
0810 00:58:35.080109 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 00:58:35.080569 29989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:35.063770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38311 (local address 127.20.81.131:43989)
0810 00:58:35.064470 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:35.064490 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:35.064653 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:35.065024 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:35.065028 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:35.065475 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:35.065754 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:35.065763 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:35.067347 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:35.067356 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:35.069485 (+ 2129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:35.069504 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:35.070500 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:35.070510 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:35.070775 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:35.071820 (+ 1045us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:35.071856 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:35.073475 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:35.077218 (+ 3743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:35.077225 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:35.077229 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:35.077747 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:35.078192 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:35.078200 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:35.078207 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:35.078384 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:35.079326 (+ 942us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:35.079330 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:35.079622 (+ 292us) client_negotiation.cc:769] Sending connection context
0810 00:58:35.079775 (+ 153us) client_negotiation.cc:240] Negotiation successful
0810 00:58:35.080127 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":381,"threads_started":1}
I20260810 00:58:35.081220 29869 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 00:58:35.083700 30026 heartbeater.cc:347] Connected to a master server at 127.20.81.190:38311
I20260810 00:58:35.083937 30026 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:35.084635 30026 heartbeater.cc:511] Master 127.20.81.190:38311 requested a full tablet report, sending...
I20260810 00:58:35.085944 29476 ts_manager.cc:194] Registered new tserver with Master: 49a07267d0814d14ba4acdf42c845584 (127.20.81.131:33811)
I20260810 00:58:35.086836 29476 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:59215
I20260810 00:58:35.089787 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 29869
I20260810 00:58:35.090001 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 00:58:35.090543 20806 external_mini_cluster.cc:1428] Setting key 044a4d98b629854bc3ad687a1c470f93
I20260810 00:58:35.093619 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 00:58:35 dist-test-slave-gp6n krb5kdc[29424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323515, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 00:58:35 dist-test-slave-gp6n krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:35 dist-test-slave-gp6n krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:35.548635 29865 heartbeater.cc:503] Master 127.20.81.190:38311 was elected leader, sending a full tablet report...
I20260810 00:58:36.089203 30026 heartbeater.cc:503] Master 127.20.81.190:38311 was elected leader, sending a full tablet report...
W20260810 00:58:36.267135 30034 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.118s user 0.008s sys 0.012s
W20260810 00:58:36.268469 29894 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:35.142886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58164 (local address 127.20.81.190:38311)
0810 00:58:35.143099 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:35.143104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:35.143537 (+ 433us) server_negotiation.cc:407] Connection header received
0810 00:58:35.143707 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:35.143712 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:35.143786 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:35.143891 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:35.145286 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:35.146075 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:35.147431 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:35.147867 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:35.147977 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.268205 (+1120228us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58164: BlockingRecv error: recv got EOF from 127.0.0.1:58164 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 10 00:58:36 dist-test-slave-gp6n krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323515, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM
I20260810 00:58:36.291234 29894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.272553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58166 (local address 127.20.81.190:38311)
0810 00:58:36.272721 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:36.272725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:36.273098 (+ 373us) server_negotiation.cc:407] Connection header received
0810 00:58:36.273266 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:36.273270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:36.273338 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:36.273435 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:36.274719 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.275447 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.276684 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.277148 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.277241 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.285762 (+ 8521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:36.285767 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:36.285770 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:36.285808 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:36.288587 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:36.289239 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:36.289243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:36.289245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:36.289374 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:36.289718 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:36.289723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:36.289726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:36.290018 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:36.290292 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:36.290613 (+ 321us) server_negotiation.cc:299] Negotiation successful
0810 00:58:36.290855 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 00:58:36.295527 29476 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:58:36.302556 29476 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:58:36.326069 30048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.315239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:45663 (local address 127.0.0.1:46750)
0810 00:58:36.317372 (+ 2133us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:36.317438 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:36.317806 (+ 368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:36.318562 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:36.318569 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:36.318599 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:36.319009 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:36.319020 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.320454 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.320465 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:36.323039 (+ 2574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:36.323056 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.325173 (+ 2117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.325179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:36.325383 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.325476 (+ 93us) client_negotiation.cc:769] Sending connection context
0810 00:58:36.325616 (+ 140us) client_negotiation.cc:240] Negotiation successful
0810 00:58:36.325782 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1793,"thread_start_us":5326,"threads_started":1}
I20260810 00:58:36.326867 30049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.315469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46750 (local address 127.20.81.129:45663)
0810 00:58:36.316191 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:36.316204 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:36.317573 (+ 1369us) server_negotiation.cc:407] Connection header received
0810 00:58:36.318127 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:36.318133 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:36.318324 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:36.318560 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:36.319192 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.320243 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.323213 (+ 2970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.324678 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.324810 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.325026 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:36.326371 (+ 1345us) server_negotiation.cc:299] Negotiation successful
0810 00:58:36.326604 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":823,"threads_started":1}
I20260810 00:58:36.329428 30047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.315399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:45399 (local address 127.0.0.1:50564)
0810 00:58:36.320878 (+ 5479us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:36.320899 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:36.321051 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:36.322181 (+ 1130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:36.322193 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:36.322241 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:36.322757 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:36.322771 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.324756 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.324761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:36.326812 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:36.326827 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.328592 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.328598 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:36.328761 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.328805 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 00:58:36.328906 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 00:58:36.329037 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5331,"mutex_wait_us":23,"thread_start_us":1465,"threads_started":1}
I20260810 00:58:36.329442 30050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.316042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50564 (local address 127.20.81.130:45399)
0810 00:58:36.316716 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:36.316727 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:36.321282 (+ 4555us) server_negotiation.cc:407] Connection header received
0810 00:58:36.321400 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:36.321407 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:36.321711 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:36.321922 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:36.322966 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.324565 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.326991 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.328383 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.328544 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.328763 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:36.328972 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 00:58:36.329138 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":456,"threads_started":1}
I20260810 00:58:36.330528 30053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.321936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:33811 (local address 127.0.0.1:59606)
0810 00:58:36.322579 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:36.322599 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:36.322728 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:36.323325 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:36.323330 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:36.323356 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:36.323684 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:36.323696 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.325620 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.325631 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:36.327561 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:36.327572 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.329734 (+ 2162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.329742 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:36.329919 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.329951 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 00:58:36.330046 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 00:58:36.330168 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":282,"threads_started":1}
I20260810 00:58:36.330543 30052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.321304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59606 (local address 127.20.81.131:33811)
0810 00:58:36.321895 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:36.321904 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:36.322716 (+ 812us) server_negotiation.cc:407] Connection header received
0810 00:58:36.322890 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:36.322900 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:36.323106 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:36.323323 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:36.324014 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.325466 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.327711 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.329081 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.329223 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.329430 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:36.330111 (+ 681us) server_negotiation.cc:299] Negotiation successful
0810 00:58:36.330237 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":281,"threads_started":1}
I20260810 00:58:36.331756 29633 tablet_service.cc:1467] Processing CreateTablet for tablet 091c4462e25b47309ffbc5319be64a36 (DEFAULT_TABLE table=test-table [id=bc3c2313f8b843dd9610cb8cff95cb24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:36.333352 29633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 091c4462e25b47309ffbc5319be64a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:36.333716 29950 tablet_service.cc:1467] Processing CreateTablet for tablet 091c4462e25b47309ffbc5319be64a36 (DEFAULT_TABLE table=test-table [id=bc3c2313f8b843dd9610cb8cff95cb24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:36.334161 29786 tablet_service.cc:1467] Processing CreateTablet for tablet 091c4462e25b47309ffbc5319be64a36 (DEFAULT_TABLE table=test-table [id=bc3c2313f8b843dd9610cb8cff95cb24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:36.335108 29950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 091c4462e25b47309ffbc5319be64a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:36.335424 29786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 091c4462e25b47309ffbc5319be64a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:36.359285 30054 tablet_bootstrap.cc:492] T 091c4462e25b47309ffbc5319be64a36 P 49a07267d0814d14ba4acdf42c845584: Bootstrap starting.
I20260810 00:58:36.358214 30056 tablet_bootstrap.cc:492] T 091c4462e25b47309ffbc5319be64a36 P 0ac5028bde95499e912b1bb99ba8a7d9: Bootstrap starting.
I20260810 00:58:36.360181 30055 tablet_bootstrap.cc:492] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a: Bootstrap starting.
I20260810 00:58:36.363368 30056 tablet_bootstrap.cc:654] T 091c4462e25b47309ffbc5319be64a36 P 0ac5028bde95499e912b1bb99ba8a7d9: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:36.363423 30055 tablet_bootstrap.cc:654] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:36.364537 30055 log.cc:824] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:36.364691 30056 log.cc:824] T 091c4462e25b47309ffbc5319be64a36 P 0ac5028bde95499e912b1bb99ba8a7d9: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:36.365746 30054 tablet_bootstrap.cc:654] T 091c4462e25b47309ffbc5319be64a36 P 49a07267d0814d14ba4acdf42c845584: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:36.366824 30054 log.cc:824] T 091c4462e25b47309ffbc5319be64a36 P 49a07267d0814d14ba4acdf42c845584: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:36.367559 30055 tablet_bootstrap.cc:492] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a: No bootstrap required, opened a new log
I20260810 00:58:36.367866 30055 ts_tablet_manager.cc:1397] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.002s
I20260810 00:58:36.369361 30054 tablet_bootstrap.cc:492] T 091c4462e25b47309ffbc5319be64a36 P 49a07267d0814d14ba4acdf42c845584: No bootstrap required, opened a new log
I20260810 00:58:36.369599 30054 ts_tablet_manager.cc:1397] T 091c4462e25b47309ffbc5319be64a36 P 49a07267d0814d14ba4acdf42c845584: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.005s
I20260810 00:58:36.372794 30056 tablet_bootstrap.cc:492] T 091c4462e25b47309ffbc5319be64a36 P 0ac5028bde95499e912b1bb99ba8a7d9: No bootstrap required, opened a new log
I20260810 00:58:36.373050 30056 ts_tablet_manager.cc:1397] T 091c4462e25b47309ffbc5319be64a36 P 0ac5028bde95499e912b1bb99ba8a7d9: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.000s
I20260810 00:58:36.377230 30054 raft_consensus.cc:348] T 091c4462e25b47309ffbc5319be64a36 P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.377590 30054 raft_consensus.cc:374] T 091c4462e25b47309ffbc5319be64a36 P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:36.377691 30054 raft_consensus.cc:729] T 091c4462e25b47309ffbc5319be64a36 P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49a07267d0814d14ba4acdf42c845584, State: Initialized, Role: FOLLOWER
I20260810 00:58:36.377533 30055 raft_consensus.cc:348] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.377956 30055 raft_consensus.cc:374] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:36.378114 30055 raft_consensus.cc:729] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82198c3537184383a48c9538ce62b11a, State: Initialized, Role: FOLLOWER
I20260810 00:58:36.378343 30054 consensus_queue.cc:260] T 091c4462e25b47309ffbc5319be64a36 P 49a07267d0814d14ba4acdf42c845584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.378718 30055 consensus_queue.cc:260] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.380283 30054 ts_tablet_manager.cc:1428] T 091c4462e25b47309ffbc5319be64a36 P 49a07267d0814d14ba4acdf42c845584: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260810 00:58:36.383814 30055 ts_tablet_manager.cc:1428] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260810 00:58:36.384614 30056 raft_consensus.cc:348] T 091c4462e25b47309ffbc5319be64a36 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.385128 30056 raft_consensus.cc:374] T 091c4462e25b47309ffbc5319be64a36 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:36.385264 30056 raft_consensus.cc:729] T 091c4462e25b47309ffbc5319be64a36 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ac5028bde95499e912b1bb99ba8a7d9, State: Initialized, Role: FOLLOWER
I20260810 00:58:36.385983 30056 consensus_queue.cc:260] T 091c4462e25b47309ffbc5319be64a36 P 0ac5028bde95499e912b1bb99ba8a7d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.388023 30056 ts_tablet_manager.cc:1428] T 091c4462e25b47309ffbc5319be64a36 P 0ac5028bde95499e912b1bb99ba8a7d9: Time spent starting tablet: real 0.015s user 0.004s sys 0.011s
I20260810 00:58:36.432704 30062 raft_consensus.cc:482] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:58:36.432979 30062 raft_consensus.cc:504] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.434671 30062 leader_election.cc:288] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ac5028bde95499e912b1bb99ba8a7d9 (127.20.81.129:45663), 49a07267d0814d14ba4acdf42c845584 (127.20.81.131:33811)
I20260810 00:58:36.441926 30049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.435131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:50745 (local address 127.20.81.129:45663)
0810 00:58:36.435329 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:36.435333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:36.435934 (+ 601us) server_negotiation.cc:407] Connection header received
0810 00:58:36.436181 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:36.436185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:36.436268 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:36.436367 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:36.437164 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.438115 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.440066 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.440933 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.441061 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.441113 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:36.441534 (+ 421us) server_negotiation.cc:299] Negotiation successful
0810 00:58:36.441672 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 00:58:36.442034 30065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.435081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:45663 (local address 127.20.81.130:50745)
0810 00:58:36.435830 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:36.435852 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:36.435995 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:36.436478 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:36.436485 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:36.436518 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:36.436999 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:36.437015 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.438246 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.438251 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:36.439901 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:36.439913 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.441148 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.441157 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:36.441332 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.441375 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 00:58:36.441484 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 00:58:36.441600 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"spinlock_wait_cycles":21760,"thread_start_us":411,"threads_started":1}
I20260810 00:58:36.442143 30066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.435937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:33811 (local address 127.20.81.130:45955)
0810 00:58:36.436595 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:36.436609 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:36.436720 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:36.437063 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:36.437066 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:36.437111 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:36.437505 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:36.437520 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.438648 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.438652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:36.440117 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:36.440131 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.441333 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.441337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:36.441474 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.441511 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 00:58:36.441604 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 00:58:36.441718 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":1,"thread_start_us":327,"threads_started":1}
I20260810 00:58:36.442023 30052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.436144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:45955 (local address 127.20.81.131:33811)
0810 00:58:36.436370 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:36.436376 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:36.436706 (+ 330us) server_negotiation.cc:407] Connection header received
0810 00:58:36.436841 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:36.436849 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:36.436941 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:36.437048 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:36.437664 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.438500 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.440264 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.441089 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.441264 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.441335 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:36.441652 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 00:58:36.441792 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 00:58:36.443291 29658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "091c4462e25b47309ffbc5319be64a36" candidate_uuid: "82198c3537184383a48c9538ce62b11a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" is_pre_election: true
I20260810 00:58:36.443290 29975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "091c4462e25b47309ffbc5319be64a36" candidate_uuid: "82198c3537184383a48c9538ce62b11a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49a07267d0814d14ba4acdf42c845584" is_pre_election: true
I20260810 00:58:36.443882 29975 raft_consensus.cc:2454] T 091c4462e25b47309ffbc5319be64a36 P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82198c3537184383a48c9538ce62b11a in term 0.
I20260810 00:58:36.443882 29658 raft_consensus.cc:2454] T 091c4462e25b47309ffbc5319be64a36 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82198c3537184383a48c9538ce62b11a in term 0.
I20260810 00:58:36.444537 29733 leader_election.cc:302] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49a07267d0814d14ba4acdf42c845584, 82198c3537184383a48c9538ce62b11a; no voters:
I20260810 00:58:36.444875 30062 raft_consensus.cc:2781] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:58:36.445045 30062 raft_consensus.cc:482] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:58:36.445191 30062 raft_consensus.cc:3037] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:36.447661 30062 raft_consensus.cc:504] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.448184 30062 leader_election.cc:288] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a [CANDIDATE]: Term 1 election: Requested vote from peers 0ac5028bde95499e912b1bb99ba8a7d9 (127.20.81.129:45663), 49a07267d0814d14ba4acdf42c845584 (127.20.81.131:33811)
I20260810 00:58:36.448601 29658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "091c4462e25b47309ffbc5319be64a36" candidate_uuid: "82198c3537184383a48c9538ce62b11a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ac5028bde95499e912b1bb99ba8a7d9"
I20260810 00:58:36.448725 29975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "091c4462e25b47309ffbc5319be64a36" candidate_uuid: "82198c3537184383a48c9538ce62b11a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49a07267d0814d14ba4acdf42c845584"
I20260810 00:58:36.448823 29658 raft_consensus.cc:3037] T 091c4462e25b47309ffbc5319be64a36 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:36.448928 29975 raft_consensus.cc:3037] T 091c4462e25b47309ffbc5319be64a36 P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:36.452302 29658 raft_consensus.cc:2454] T 091c4462e25b47309ffbc5319be64a36 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82198c3537184383a48c9538ce62b11a in term 1.
I20260810 00:58:36.452302 29975 raft_consensus.cc:2454] T 091c4462e25b47309ffbc5319be64a36 P 49a07267d0814d14ba4acdf42c845584 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82198c3537184383a48c9538ce62b11a in term 1.
I20260810 00:58:36.452675 29733 leader_election.cc:302] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49a07267d0814d14ba4acdf42c845584, 82198c3537184383a48c9538ce62b11a; no voters:
I20260810 00:58:36.452937 30062 raft_consensus.cc:2781] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:58:36.453370 30062 raft_consensus.cc:686] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a [term 1 LEADER]: Becoming Leader. State: Replica: 82198c3537184383a48c9538ce62b11a, State: Running, Role: LEADER
I20260810 00:58:36.453838 30062 consensus_queue.cc:237] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.459021 29475 catalog_manager.cc:5495] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a reported cstate change: term changed from 0 to 1, leader changed from <none> to 82198c3537184383a48c9538ce62b11a (127.20.81.130). New cstate: current_term: 1 leader_uuid: "82198c3537184383a48c9538ce62b11a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } health_report { overall_health: UNKNOWN } } }
I20260810 00:58:36.487353 30050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.482556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50578 (local address 127.20.81.130:45399)
0810 00:58:36.482738 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:36.482743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:36.482761 (+ 18us) server_negotiation.cc:407] Connection header received
0810 00:58:36.482846 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:36.482849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:36.482928 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:36.483053 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:58:36.483354 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.484069 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.485321 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.485753 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.485846 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.486282 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:58:36.486901 (+ 619us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:58:36.487012 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:36.487091 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 00:58:36.487156 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260810 00:58:36.488883 29710 tablet.cc:2369] T 091c4462e25b47309ffbc5319be64a36 P 0ac5028bde95499e912b1bb99ba8a7d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:58:36.496073 29975 raft_consensus.cc:1261] T 091c4462e25b47309ffbc5319be64a36 P 49a07267d0814d14ba4acdf42c845584 [term 1 FOLLOWER]: Refusing update from remote peer 82198c3537184383a48c9538ce62b11a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:58:36.496074 29658 raft_consensus.cc:1261] T 091c4462e25b47309ffbc5319be64a36 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 1 FOLLOWER]: Refusing update from remote peer 82198c3537184383a48c9538ce62b11a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:58:36.497074 30067 consensus_queue.cc:1035] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:58:36.497506 30062 consensus_queue.cc:1035] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a [LEADER]: Connected to new peer: Peer: permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:58:36.509073 30070 mvcc.cc:205] Tried to move back new op lower bound from 7316781123557707776 to 7316781123398545408. Current Snapshot: MvccSnapshot[applied={T|T < 7316781123557707776}]
I20260810 00:58:36.522718 29475 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:58:36.522998 29475 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:58:36.526635 29475 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4: Sending DeleteTablet for 3 replicas of tablet 091c4462e25b47309ffbc5319be64a36
I20260810 00:58:36.527817 29786 tablet_service.cc:1514] Processing DeleteTablet for tablet 091c4462e25b47309ffbc5319be64a36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:36 UTC) from {username='oryx', principal='oryx/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:50564
I20260810 00:58:36.527879 29950 tablet_service.cc:1514] Processing DeleteTablet for tablet 091c4462e25b47309ffbc5319be64a36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:36 UTC) from {username='oryx', principal='oryx/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:59606
I20260810 00:58:36.527910 29633 tablet_service.cc:1514] Processing DeleteTablet for tablet 091c4462e25b47309ffbc5319be64a36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:36 UTC) from {username='oryx', principal='oryx/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:46750
I20260810 00:58:36.535347 29475 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:58:36.537317 29475 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:58:36.543674 30082 tablet_replica.cc:330] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a: stopping tablet replica
I20260810 00:58:36.544266 30082 raft_consensus.cc:2229] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:58:36.544411 30083 tablet_replica.cc:330] T 091c4462e25b47309ffbc5319be64a36 P 0ac5028bde95499e912b1bb99ba8a7d9: stopping tablet replica
I20260810 00:58:36.544955 30083 raft_consensus.cc:2229] T 091c4462e25b47309ffbc5319be64a36 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:58:36.545094 30082 raft_consensus.cc:2258] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:36.545557 30083 raft_consensus.cc:2258] T 091c4462e25b47309ffbc5319be64a36 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:36.547421 29949 tablet_service.cc:1467] Processing CreateTablet for tablet 011e93d8088a47a2a60db084362c373c (DEFAULT_TABLE table=test-table [id=827ff717cd914ec5b02230f09ab52b3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:36.548017 29949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 011e93d8088a47a2a60db084362c373c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:36.548343 29786 tablet_service.cc:1467] Processing CreateTablet for tablet 011e93d8088a47a2a60db084362c373c (DEFAULT_TABLE table=test-table [id=827ff717cd914ec5b02230f09ab52b3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:36.548892 29786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 011e93d8088a47a2a60db084362c373c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:36.554904 29633 tablet_service.cc:1467] Processing CreateTablet for tablet 011e93d8088a47a2a60db084362c373c (DEFAULT_TABLE table=test-table [id=827ff717cd914ec5b02230f09ab52b3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:36.555408 29633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 011e93d8088a47a2a60db084362c373c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:36.555779 30082 ts_tablet_manager.cc:1905] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:36.558818 30054 tablet_bootstrap.cc:492] T 011e93d8088a47a2a60db084362c373c P 49a07267d0814d14ba4acdf42c845584: Bootstrap starting.
I20260810 00:58:36.559504 30081 tablet_replica.cc:330] T 091c4462e25b47309ffbc5319be64a36 P 49a07267d0814d14ba4acdf42c845584: stopping tablet replica
I20260810 00:58:36.559947 30081 raft_consensus.cc:2229] T 091c4462e25b47309ffbc5319be64a36 P 49a07267d0814d14ba4acdf42c845584 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:58:36.560657 30054 tablet_bootstrap.cc:654] T 011e93d8088a47a2a60db084362c373c P 49a07267d0814d14ba4acdf42c845584: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:36.561559 30081 raft_consensus.cc:2258] T 091c4462e25b47309ffbc5319be64a36 P 49a07267d0814d14ba4acdf42c845584 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:36.563123 30083 ts_tablet_manager.cc:1905] T 091c4462e25b47309ffbc5319be64a36 P 0ac5028bde95499e912b1bb99ba8a7d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:36.564261 30081 ts_tablet_manager.cc:1905] T 091c4462e25b47309ffbc5319be64a36 P 49a07267d0814d14ba4acdf42c845584: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:36.567822 30055 tablet_bootstrap.cc:492] T 011e93d8088a47a2a60db084362c373c P 82198c3537184383a48c9538ce62b11a: Bootstrap starting.
I20260810 00:58:36.569589 30055 tablet_bootstrap.cc:654] T 011e93d8088a47a2a60db084362c373c P 82198c3537184383a48c9538ce62b11a: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:36.572854 30056 tablet_bootstrap.cc:492] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9: Bootstrap starting.
I20260810 00:58:36.573437 30083 ts_tablet_manager.cc:1918] T 091c4462e25b47309ffbc5319be64a36 P 0ac5028bde95499e912b1bb99ba8a7d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:58:36.574030 30083 log.cc:1196] T 091c4462e25b47309ffbc5319be64a36 P 0ac5028bde95499e912b1bb99ba8a7d9: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-0/wal/wals/091c4462e25b47309ffbc5319be64a36
I20260810 00:58:36.574608 30056 tablet_bootstrap.cc:654] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:36.574664 30083 ts_tablet_manager.cc:1939] T 091c4462e25b47309ffbc5319be64a36 P 0ac5028bde95499e912b1bb99ba8a7d9: Deleting consensus metadata
I20260810 00:58:36.576889 29455 catalog_manager.cc:4841] TS 0ac5028bde95499e912b1bb99ba8a7d9 (127.20.81.129:45663): tablet 091c4462e25b47309ffbc5319be64a36 (table test-table [id=bc3c2313f8b843dd9610cb8cff95cb24]) successfully deleted
I20260810 00:58:36.579013 30056 tablet_bootstrap.cc:492] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9: No bootstrap required, opened a new log
I20260810 00:58:36.579237 30056 ts_tablet_manager.cc:1397] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s
I20260810 00:58:36.579648 30054 tablet_bootstrap.cc:492] T 011e93d8088a47a2a60db084362c373c P 49a07267d0814d14ba4acdf42c845584: No bootstrap required, opened a new log
I20260810 00:58:36.579800 30054 ts_tablet_manager.cc:1397] T 011e93d8088a47a2a60db084362c373c P 49a07267d0814d14ba4acdf42c845584: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.001s
I20260810 00:58:36.580374 30056 raft_consensus.cc:348] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.580665 30056 raft_consensus.cc:374] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:36.580729 30054 raft_consensus.cc:348] T 011e93d8088a47a2a60db084362c373c P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.580863 30056 raft_consensus.cc:729] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ac5028bde95499e912b1bb99ba8a7d9, State: Initialized, Role: FOLLOWER
I20260810 00:58:36.580960 30054 raft_consensus.cc:374] T 011e93d8088a47a2a60db084362c373c P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:36.581049 30054 raft_consensus.cc:729] T 011e93d8088a47a2a60db084362c373c P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49a07267d0814d14ba4acdf42c845584, State: Initialized, Role: FOLLOWER
I20260810 00:58:36.581385 30056 consensus_queue.cc:260] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.581408 30054 consensus_queue.cc:260] T 011e93d8088a47a2a60db084362c373c P 49a07267d0814d14ba4acdf42c845584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.583083 30056 ts_tablet_manager.cc:1428] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260810 00:58:36.583572 30054 ts_tablet_manager.cc:1428] T 011e93d8088a47a2a60db084362c373c P 49a07267d0814d14ba4acdf42c845584: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260810 00:58:36.583808 30081 ts_tablet_manager.cc:1918] T 091c4462e25b47309ffbc5319be64a36 P 49a07267d0814d14ba4acdf42c845584: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:58:36.583984 30081 log.cc:1196] T 091c4462e25b47309ffbc5319be64a36 P 49a07267d0814d14ba4acdf42c845584: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/091c4462e25b47309ffbc5319be64a36
W20260810 00:58:36.584560 30027 tablet.cc:2369] T 011e93d8088a47a2a60db084362c373c P 49a07267d0814d14ba4acdf42c845584: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:58:36.584635 30081 ts_tablet_manager.cc:1939] T 091c4462e25b47309ffbc5319be64a36 P 49a07267d0814d14ba4acdf42c845584: Deleting consensus metadata
I20260810 00:58:36.586666 29455 catalog_manager.cc:4841] TS 49a07267d0814d14ba4acdf42c845584 (127.20.81.131:33811): tablet 091c4462e25b47309ffbc5319be64a36 (table test-table [id=bc3c2313f8b843dd9610cb8cff95cb24]) successfully deleted
I20260810 00:58:36.589164 30055 tablet_bootstrap.cc:492] T 011e93d8088a47a2a60db084362c373c P 82198c3537184383a48c9538ce62b11a: No bootstrap required, opened a new log
I20260810 00:58:36.589466 30055 ts_tablet_manager.cc:1397] T 011e93d8088a47a2a60db084362c373c P 82198c3537184383a48c9538ce62b11a: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.002s
I20260810 00:58:36.590478 30055 raft_consensus.cc:348] T 011e93d8088a47a2a60db084362c373c P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.590761 30055 raft_consensus.cc:374] T 011e93d8088a47a2a60db084362c373c P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:36.590868 30055 raft_consensus.cc:729] T 011e93d8088a47a2a60db084362c373c P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82198c3537184383a48c9538ce62b11a, State: Initialized, Role: FOLLOWER
I20260810 00:58:36.591182 30055 consensus_queue.cc:260] T 011e93d8088a47a2a60db084362c373c P 82198c3537184383a48c9538ce62b11a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.591763 30055 ts_tablet_manager.cc:1428] T 011e93d8088a47a2a60db084362c373c P 82198c3537184383a48c9538ce62b11a: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260810 00:58:36.592304 30082 ts_tablet_manager.cc:1918] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:58:36.592506 30082 log.cc:1196] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/091c4462e25b47309ffbc5319be64a36
I20260810 00:58:36.593142 30082 ts_tablet_manager.cc:1939] T 091c4462e25b47309ffbc5319be64a36 P 82198c3537184383a48c9538ce62b11a: Deleting consensus metadata
I20260810 00:58:36.595052 29457 catalog_manager.cc:4841] TS 82198c3537184383a48c9538ce62b11a (127.20.81.130:45399): tablet 091c4462e25b47309ffbc5319be64a36 (table test-table [id=bc3c2313f8b843dd9610cb8cff95cb24]) successfully deleted
I20260810 00:58:36.628739 30063 raft_consensus.cc:482] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:58:36.628985 30063 raft_consensus.cc:504] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.630779 30063 leader_election.cc:288] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82198c3537184383a48c9538ce62b11a (127.20.81.130:45399), 49a07267d0814d14ba4acdf42c845584 (127.20.81.131:33811)
I20260810 00:58:36.638545 30052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.631567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:42461 (local address 127.20.81.131:33811)
0810 00:58:36.631747 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:36.631753 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:36.632472 (+ 719us) server_negotiation.cc:407] Connection header received
0810 00:58:36.632592 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:36.632599 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:36.632674 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:36.632764 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:36.633539 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.634632 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.636381 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.637287 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.637452 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.637532 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:36.638088 (+ 556us) server_negotiation.cc:299] Negotiation successful
0810 00:58:36.638230 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 00:58:36.639216 30089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.631106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:33811 (local address 127.20.81.129:42461)
0810 00:58:36.632357 (+ 1251us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:36.632374 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:36.632484 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:36.632814 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:36.632823 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:36.632850 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:36.633337 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:36.633351 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.634739 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.634745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:36.636224 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:36.636236 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.637680 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.637686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:36.637884 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.637918 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 00:58:36.638617 (+ 699us) client_negotiation.cc:240] Negotiation successful
0810 00:58:36.638757 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1120,"mutex_wait_us":29,"thread_start_us":903,"threads_started":1}
I20260810 00:58:36.639671 30050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.631276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:42041 (local address 127.20.81.130:45399)
0810 00:58:36.631525 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:36.631534 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:36.632132 (+ 598us) server_negotiation.cc:407] Connection header received
0810 00:58:36.632439 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:36.632446 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:36.632525 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:36.632624 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:36.633256 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.634545 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.637085 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.638437 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.638619 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.638705 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:36.639231 (+ 526us) server_negotiation.cc:299] Negotiation successful
0810 00:58:36.639365 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 00:58:36.639719 30088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.631003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:45399 (local address 127.20.81.129:42041)
0810 00:58:36.632011 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:36.632034 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:36.632257 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:36.632637 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:36.632641 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:36.632663 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:36.633093 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:36.633105 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.634699 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.634705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:36.636912 (+ 2207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:36.636929 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.638871 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.638878 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:36.639062 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.639086 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 00:58:36.639200 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 00:58:36.639323 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":858,"mutex_wait_us":2,"thread_start_us":706,"threads_started":1}
I20260810 00:58:36.640075 29975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "011e93d8088a47a2a60db084362c373c" candidate_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49a07267d0814d14ba4acdf42c845584" is_pre_election: true
I20260810 00:58:36.640370 29975 raft_consensus.cc:2454] T 011e93d8088a47a2a60db084362c373c P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ac5028bde95499e912b1bb99ba8a7d9 in term 0.
I20260810 00:58:36.640869 29574 leader_election.cc:302] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ac5028bde95499e912b1bb99ba8a7d9, 49a07267d0814d14ba4acdf42c845584; no voters:
I20260810 00:58:36.640806 29813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "011e93d8088a47a2a60db084362c373c" candidate_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82198c3537184383a48c9538ce62b11a" is_pre_election: true
I20260810 00:58:36.641268 30063 raft_consensus.cc:2781] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:58:36.641276 29813 raft_consensus.cc:2454] T 011e93d8088a47a2a60db084362c373c P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ac5028bde95499e912b1bb99ba8a7d9 in term 0.
I20260810 00:58:36.641396 30063 raft_consensus.cc:482] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:58:36.641512 30063 raft_consensus.cc:3037] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:36.643800 30063 raft_consensus.cc:504] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.644333 30063 leader_election.cc:288] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9 [CANDIDATE]: Term 1 election: Requested vote from peers 82198c3537184383a48c9538ce62b11a (127.20.81.130:45399), 49a07267d0814d14ba4acdf42c845584 (127.20.81.131:33811)
I20260810 00:58:36.644793 29813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "011e93d8088a47a2a60db084362c373c" candidate_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82198c3537184383a48c9538ce62b11a"
I20260810 00:58:36.644886 29975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "011e93d8088a47a2a60db084362c373c" candidate_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49a07267d0814d14ba4acdf42c845584"
I20260810 00:58:36.645011 29813 raft_consensus.cc:3037] T 011e93d8088a47a2a60db084362c373c P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:36.645108 29975 raft_consensus.cc:3037] T 011e93d8088a47a2a60db084362c373c P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:36.647356 29813 raft_consensus.cc:2454] T 011e93d8088a47a2a60db084362c373c P 82198c3537184383a48c9538ce62b11a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ac5028bde95499e912b1bb99ba8a7d9 in term 1.
I20260810 00:58:36.647773 29576 leader_election.cc:302] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ac5028bde95499e912b1bb99ba8a7d9, 82198c3537184383a48c9538ce62b11a; no voters:
I20260810 00:58:36.648077 30063 raft_consensus.cc:2781] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:58:36.648142 29975 raft_consensus.cc:2454] T 011e93d8088a47a2a60db084362c373c P 49a07267d0814d14ba4acdf42c845584 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ac5028bde95499e912b1bb99ba8a7d9 in term 1.
I20260810 00:58:36.648804 30063 raft_consensus.cc:686] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 1 LEADER]: Becoming Leader. State: Replica: 0ac5028bde95499e912b1bb99ba8a7d9, State: Running, Role: LEADER
I20260810 00:58:36.649415 30063 consensus_queue.cc:237] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.653182 29475 catalog_manager.cc:5495] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ac5028bde95499e912b1bb99ba8a7d9 (127.20.81.129). New cstate: current_term: 1 leader_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } health_report { overall_health: UNKNOWN } } }
Aug 10 00:58:36 dist-test-slave-gp6n krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323513, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.190@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM
I20260810 00:58:36.716737 30098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.700964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38311 (local address 127.0.0.1:58172)
0810 00:58:36.701637 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:36.701669 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:36.701797 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:36.702176 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:36.702179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:36.702953 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:36.703220 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:36.703229 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.704728 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.704731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:36.705699 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:36.705711 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.706668 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.706672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:36.706843 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.707585 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:36.707607 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:36.710270 (+ 2663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:36.714025 (+ 3755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:36.714035 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:36.714041 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:36.714493 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:36.714895 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:36.714899 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:36.714901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:36.715051 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:36.715703 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:36.715709 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:36.715933 (+ 224us) client_negotiation.cc:769] Sending connection context
0810 00:58:36.716109 (+ 176us) client_negotiation.cc:240] Negotiation successful
0810 00:58:36.716410 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":375,"threads_started":1}
I20260810 00:58:36.717012 29894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.700939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58172 (local address 127.20.81.190:38311)
0810 00:58:36.701173 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:36.701180 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:36.701769 (+ 589us) server_negotiation.cc:407] Connection header received
0810 00:58:36.701917 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:36.701923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:36.702006 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:36.702124 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:36.703370 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.704583 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.705852 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.706502 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.706629 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.710497 (+ 3868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:36.710503 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:36.710508 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:36.710548 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:36.713879 (+ 3331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:36.714602 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:36.714608 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:36.714612 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:36.714751 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:36.715205 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:36.715211 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:36.715214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:36.715574 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:36.715727 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:36.716237 (+ 510us) server_negotiation.cc:299] Negotiation successful
0810 00:58:36.716603 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":78}
I20260810 00:58:36.722122 29475 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:58172:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 00:58:36.723266 29475 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:58:36.730288 29633 tablet_service.cc:1467] Processing CreateTablet for tablet 26ea962f6e8c4b1795c38e209de22f7c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=30224ee3481a488bb614b00517bc0148]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 00:58:36.730518 29950 tablet_service.cc:1467] Processing CreateTablet for tablet 26ea962f6e8c4b1795c38e209de22f7c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=30224ee3481a488bb614b00517bc0148]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 00:58:36.730937 29633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26ea962f6e8c4b1795c38e209de22f7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:36.731184 29950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26ea962f6e8c4b1795c38e209de22f7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:36.735579 29786 tablet_service.cc:1467] Processing CreateTablet for tablet 26ea962f6e8c4b1795c38e209de22f7c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=30224ee3481a488bb614b00517bc0148]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 00:58:36.736155 29786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26ea962f6e8c4b1795c38e209de22f7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:36.736814 30054 tablet_bootstrap.cc:492] T 26ea962f6e8c4b1795c38e209de22f7c P 49a07267d0814d14ba4acdf42c845584: Bootstrap starting.
I20260810 00:58:36.738571 30054 tablet_bootstrap.cc:654] T 26ea962f6e8c4b1795c38e209de22f7c P 49a07267d0814d14ba4acdf42c845584: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:36.740947 30054 tablet_bootstrap.cc:492] T 26ea962f6e8c4b1795c38e209de22f7c P 49a07267d0814d14ba4acdf42c845584: No bootstrap required, opened a new log
I20260810 00:58:36.741179 30054 ts_tablet_manager.cc:1397] T 26ea962f6e8c4b1795c38e209de22f7c P 49a07267d0814d14ba4acdf42c845584: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20260810 00:58:36.741945 30054 raft_consensus.cc:348] T 26ea962f6e8c4b1795c38e209de22f7c P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.742201 30054 raft_consensus.cc:374] T 26ea962f6e8c4b1795c38e209de22f7c P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:36.742336 30054 raft_consensus.cc:729] T 26ea962f6e8c4b1795c38e209de22f7c P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49a07267d0814d14ba4acdf42c845584, State: Initialized, Role: FOLLOWER
I20260810 00:58:36.742626 30054 consensus_queue.cc:260] T 26ea962f6e8c4b1795c38e209de22f7c P 49a07267d0814d14ba4acdf42c845584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.743331 30055 tablet_bootstrap.cc:492] T 26ea962f6e8c4b1795c38e209de22f7c P 82198c3537184383a48c9538ce62b11a: Bootstrap starting.
I20260810 00:58:36.743129 30061 tablet_replica.cc:439] T 26ea962f6e8c4b1795c38e209de22f7c P 49a07267d0814d14ba4acdf42c845584: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } } }
I20260810 00:58:36.743492 30061 tablet_replica.cc:442] T 26ea962f6e8c4b1795c38e209de22f7c P 49a07267d0814d14ba4acdf42c845584: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:58:36.744488 30054 ts_tablet_manager.cc:1428] T 26ea962f6e8c4b1795c38e209de22f7c P 49a07267d0814d14ba4acdf42c845584: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260810 00:58:36.744964 30055 tablet_bootstrap.cc:654] T 26ea962f6e8c4b1795c38e209de22f7c P 82198c3537184383a48c9538ce62b11a: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:36.745916 30056 tablet_bootstrap.cc:492] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9: Bootstrap starting.
I20260810 00:58:36.747853 30056 tablet_bootstrap.cc:654] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:36.755568 30056 tablet_bootstrap.cc:492] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9: No bootstrap required, opened a new log
I20260810 00:58:36.755777 30056 ts_tablet_manager.cc:1397] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260810 00:58:36.756479 30055 tablet_bootstrap.cc:492] T 26ea962f6e8c4b1795c38e209de22f7c P 82198c3537184383a48c9538ce62b11a: No bootstrap required, opened a new log
I20260810 00:58:36.756673 30055 ts_tablet_manager.cc:1397] T 26ea962f6e8c4b1795c38e209de22f7c P 82198c3537184383a48c9538ce62b11a: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.010s
I20260810 00:58:36.756583 30056 raft_consensus.cc:348] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.756888 30056 raft_consensus.cc:374] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:36.756989 30056 raft_consensus.cc:729] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ac5028bde95499e912b1bb99ba8a7d9, State: Initialized, Role: FOLLOWER
I20260810 00:58:36.757319 30056 consensus_queue.cc:260] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.757465 30055 raft_consensus.cc:348] T 26ea962f6e8c4b1795c38e209de22f7c P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.757714 30055 raft_consensus.cc:374] T 26ea962f6e8c4b1795c38e209de22f7c P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:36.757815 30055 raft_consensus.cc:729] T 26ea962f6e8c4b1795c38e209de22f7c P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82198c3537184383a48c9538ce62b11a, State: Initialized, Role: FOLLOWER
I20260810 00:58:36.757751 30063 tablet_replica.cc:439] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } } }
I20260810 00:58:36.758049 30063 tablet_replica.cc:442] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:58:36.758132 30055 consensus_queue.cc:260] T 26ea962f6e8c4b1795c38e209de22f7c P 82198c3537184383a48c9538ce62b11a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.758602 30067 tablet_replica.cc:439] T 26ea962f6e8c4b1795c38e209de22f7c P 82198c3537184383a48c9538ce62b11a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } } }
I20260810 00:58:36.758972 30067 tablet_replica.cc:442] T 26ea962f6e8c4b1795c38e209de22f7c P 82198c3537184383a48c9538ce62b11a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:58:36.759353 30056 ts_tablet_manager.cc:1428] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260810 00:58:36.761394 30055 ts_tablet_manager.cc:1428] T 26ea962f6e8c4b1795c38e209de22f7c P 82198c3537184383a48c9538ce62b11a: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
W20260810 00:58:36.792953 29866 tablet.cc:2369] T 011e93d8088a47a2a60db084362c373c P 82198c3537184383a48c9538ce62b11a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:58:36.820753 30063 raft_consensus.cc:482] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:58:36.820998 30063 raft_consensus.cc:504] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.821661 30063 leader_election.cc:288] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82198c3537184383a48c9538ce62b11a (127.20.81.130:45399), 49a07267d0814d14ba4acdf42c845584 (127.20.81.131:33811)
I20260810 00:58:36.822250 29975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ea962f6e8c4b1795c38e209de22f7c" candidate_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49a07267d0814d14ba4acdf42c845584" is_pre_election: true
I20260810 00:58:36.822286 29813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ea962f6e8c4b1795c38e209de22f7c" candidate_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82198c3537184383a48c9538ce62b11a" is_pre_election: true
I20260810 00:58:36.822567 29975 raft_consensus.cc:2454] T 26ea962f6e8c4b1795c38e209de22f7c P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ac5028bde95499e912b1bb99ba8a7d9 in term 0.
I20260810 00:58:36.822614 29813 raft_consensus.cc:2454] T 26ea962f6e8c4b1795c38e209de22f7c P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ac5028bde95499e912b1bb99ba8a7d9 in term 0.
I20260810 00:58:36.823022 29574 leader_election.cc:302] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ac5028bde95499e912b1bb99ba8a7d9, 49a07267d0814d14ba4acdf42c845584; no voters:
I20260810 00:58:36.823320 30063 raft_consensus.cc:2781] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:58:36.823433 30063 raft_consensus.cc:482] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:58:36.823553 30063 raft_consensus.cc:3037] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:36.825654 30063 raft_consensus.cc:504] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.826229 30063 leader_election.cc:288] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9 [CANDIDATE]: Term 1 election: Requested vote from peers 82198c3537184383a48c9538ce62b11a (127.20.81.130:45399), 49a07267d0814d14ba4acdf42c845584 (127.20.81.131:33811)
I20260810 00:58:36.826701 29975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ea962f6e8c4b1795c38e209de22f7c" candidate_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49a07267d0814d14ba4acdf42c845584"
I20260810 00:58:36.826733 29813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ea962f6e8c4b1795c38e209de22f7c" candidate_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82198c3537184383a48c9538ce62b11a"
I20260810 00:58:36.826930 29975 raft_consensus.cc:3037] T 26ea962f6e8c4b1795c38e209de22f7c P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:36.826956 29813 raft_consensus.cc:3037] T 26ea962f6e8c4b1795c38e209de22f7c P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:36.828975 29975 raft_consensus.cc:2454] T 26ea962f6e8c4b1795c38e209de22f7c P 49a07267d0814d14ba4acdf42c845584 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ac5028bde95499e912b1bb99ba8a7d9 in term 1.
I20260810 00:58:36.829483 29574 leader_election.cc:302] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ac5028bde95499e912b1bb99ba8a7d9, 49a07267d0814d14ba4acdf42c845584; no voters:
I20260810 00:58:36.829772 30063 raft_consensus.cc:2781] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:58:36.830155 30063 raft_consensus.cc:686] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 1 LEADER]: Becoming Leader. State: Replica: 0ac5028bde95499e912b1bb99ba8a7d9, State: Running, Role: LEADER
I20260810 00:58:36.830238 29813 raft_consensus.cc:2454] T 26ea962f6e8c4b1795c38e209de22f7c P 82198c3537184383a48c9538ce62b11a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ac5028bde95499e912b1bb99ba8a7d9 in term 1.
I20260810 00:58:36.830519 30063 consensus_queue.cc:237] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:36.832465 30091 tablet_replica.cc:439] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9: TxnStatusTablet state changed. Reason: New leader 0ac5028bde95499e912b1bb99ba8a7d9. Latest consensus state: current_term: 1 leader_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } } }
I20260810 00:58:36.832919 30091 tablet_replica.cc:442] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:58:36.833618 30104 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 00:58:36.833856 30104 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 00:58:36.834091 29475 catalog_manager.cc:5495] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ac5028bde95499e912b1bb99ba8a7d9 (127.20.81.129). New cstate: current_term: 1 leader_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } health_report { overall_health: UNKNOWN } } }
I20260810 00:58:36.855551 30098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.849763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:45663 (local address 127.0.0.1:46762)
0810 00:58:36.850077 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:36.850093 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:36.850255 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:36.850739 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:36.850744 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:36.850757 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:58:36.850971 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:36.850980 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.852035 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.852042 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:36.853066 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:36.853078 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.853813 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.853818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:36.853997 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.854055 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:58:36.855007 (+ 952us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:58:36.855030 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 00:58:36.855117 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 00:58:36.855268 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158}
I20260810 00:58:36.855588 30049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.849779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46762 (local address 127.20.81.129:45663)
0810 00:58:36.849952 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:36.849959 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:36.850167 (+ 208us) server_negotiation.cc:407] Connection header received
0810 00:58:36.850481 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:36.850486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:36.850572 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:36.850716 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:58:36.851105 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.851896 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.853208 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.853651 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.853787 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.854293 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:58:36.854853 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:58:36.855039 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:36.855138 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 00:58:36.855224 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 00:58:36.860091 29975 raft_consensus.cc:1261] T 26ea962f6e8c4b1795c38e209de22f7c P 49a07267d0814d14ba4acdf42c845584 [term 1 FOLLOWER]: Refusing update from remote peer 0ac5028bde95499e912b1bb99ba8a7d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:58:36.860536 29813 raft_consensus.cc:1261] T 26ea962f6e8c4b1795c38e209de22f7c P 82198c3537184383a48c9538ce62b11a [term 1 FOLLOWER]: Refusing update from remote peer 0ac5028bde95499e912b1bb99ba8a7d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:58:36.861052 30063 consensus_queue.cc:1035] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:58:36.861413 30091 consensus_queue.cc:1035] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:58:36.870369 30067 tablet_replica.cc:439] T 26ea962f6e8c4b1795c38e209de22f7c P 82198c3537184383a48c9538ce62b11a: TxnStatusTablet state changed. Reason: New leader 0ac5028bde95499e912b1bb99ba8a7d9. Latest consensus state: current_term: 1 leader_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } } }
I20260810 00:58:36.870813 30067 tablet_replica.cc:442] T 26ea962f6e8c4b1795c38e209de22f7c P 82198c3537184383a48c9538ce62b11a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:58:36.875732 30061 tablet_replica.cc:439] T 26ea962f6e8c4b1795c38e209de22f7c P 49a07267d0814d14ba4acdf42c845584: TxnStatusTablet state changed. Reason: New leader 0ac5028bde95499e912b1bb99ba8a7d9. Latest consensus state: current_term: 1 leader_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } } }
I20260810 00:58:36.876050 30061 tablet_replica.cc:442] T 26ea962f6e8c4b1795c38e209de22f7c P 49a07267d0814d14ba4acdf42c845584: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:58:36.876262 30063 tablet_replica.cc:439] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } } }
I20260810 00:58:36.876560 30063 tablet_replica.cc:442] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:58:36.876835 30063 tablet_replica.cc:439] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } } }
I20260810 00:58:36.877094 30063 tablet_replica.cc:442] T 26ea962f6e8c4b1795c38e209de22f7c P 0ac5028bde95499e912b1bb99ba8a7d9: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:58:36.878032 30072 mvcc.cc:205] Tried to move back new op lower bound from 7316781125050564608 to 7316781124940718080. Current Snapshot: MvccSnapshot[applied={T|T < 7316781125050564608 or (T in {7316781125050564608})}]
I20260810 00:58:36.881021 30067 tablet_replica.cc:439] T 26ea962f6e8c4b1795c38e209de22f7c P 82198c3537184383a48c9538ce62b11a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } } }
I20260810 00:58:36.881304 30067 tablet_replica.cc:442] T 26ea962f6e8c4b1795c38e209de22f7c P 82198c3537184383a48c9538ce62b11a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:58:36.887601 30061 tablet_replica.cc:439] T 26ea962f6e8c4b1795c38e209de22f7c P 49a07267d0814d14ba4acdf42c845584: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } } }
I20260810 00:58:36.887848 30061 tablet_replica.cc:442] T 26ea962f6e8c4b1795c38e209de22f7c P 49a07267d0814d14ba4acdf42c845584: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:58:36.891239 30049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.885142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46776 (local address 127.20.81.129:45663)
0810 00:58:36.885312 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:36.885316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:36.885330 (+ 14us) server_negotiation.cc:407] Connection header received
0810 00:58:36.885377 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:36.885381 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:36.885435 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:36.885549 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:58:36.886020 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.887153 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.889068 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.889696 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.889814 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.890365 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:58:36.890543 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:58:36.890640 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:36.890784 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 00:58:36.890858 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 00:58:36.907958 30049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.901819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:42615 (local address 127.20.81.129:45663)
0810 00:58:36.902005 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:36.902012 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:36.902794 (+ 782us) server_negotiation.cc:407] Connection header received
0810 00:58:36.903005 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:36.903010 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:36.903085 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:36.903187 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:58:36.903718 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.904476 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.905900 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.906477 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.906616 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.907046 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:58:36.907183 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:58:36.907285 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:36.907431 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 00:58:36.907522 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 00:58:36.908015 30111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:36.901906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:45663 (local address 127.20.81.129:42615)
0810 00:58:36.902688 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:36.902709 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:36.902883 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:36.903198 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:36.903202 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:36.903211 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:58:36.903562 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:36.903571 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.904608 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.904613 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:36.905755 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:36.905767 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:36.906670 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:36.906678 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:36.906820 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:36.906881 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:58:36.907280 (+ 399us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:58:36.907301 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 00:58:36.907402 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 00:58:36.907596 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":408,"threads_started":1}
I20260810 00:58:36.921825 29808 raft_consensus.cc:1261] T 011e93d8088a47a2a60db084362c373c P 82198c3537184383a48c9538ce62b11a [term 1 FOLLOWER]: Refusing update from remote peer 0ac5028bde95499e912b1bb99ba8a7d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:58:36.922372 30091 consensus_queue.cc:1035] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:58:36.922616 29975 raft_consensus.cc:1261] T 011e93d8088a47a2a60db084362c373c P 49a07267d0814d14ba4acdf42c845584 [term 1 FOLLOWER]: Refusing update from remote peer 0ac5028bde95499e912b1bb99ba8a7d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:58:36.923182 30091 consensus_queue.cc:1035] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:58:36.943641 30071 mvcc.cc:205] Tried to move back new op lower bound from 7316781125306155008 to 7316781124199657472. Current Snapshot: MvccSnapshot[applied={T|T < 7316781125306155008}]
W20260810 00:58:37.007208 29574 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.81.131:33811, user_credentials={real_user=slave}} blocked reactor thread for 54331.2us
I20260810 00:58:37.027320 29475 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:58:37.027594 29475 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:58:37.031102 29475 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4: Sending DeleteTablet for 3 replicas of tablet 011e93d8088a47a2a60db084362c373c
I20260810 00:58:37.032112 29633 tablet_service.cc:1514] Processing DeleteTablet for tablet 011e93d8088a47a2a60db084362c373c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:37 UTC) from {username='oryx', principal='oryx/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:46750
I20260810 00:58:37.032410 30083 tablet_replica.cc:330] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9: stopping tablet replica
I20260810 00:58:37.032507 29786 tablet_service.cc:1514] Processing DeleteTablet for tablet 011e93d8088a47a2a60db084362c373c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:37 UTC) from {username='oryx', principal='oryx/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:50564
I20260810 00:58:37.032662 30083 raft_consensus.cc:2229] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:58:37.032786 30082 tablet_replica.cc:330] T 011e93d8088a47a2a60db084362c373c P 82198c3537184383a48c9538ce62b11a: stopping tablet replica
I20260810 00:58:37.032992 30082 raft_consensus.cc:2229] T 011e93d8088a47a2a60db084362c373c P 82198c3537184383a48c9538ce62b11a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:58:37.033200 30083 raft_consensus.cc:2258] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:37.033227 30082 raft_consensus.cc:2258] T 011e93d8088a47a2a60db084362c373c P 82198c3537184383a48c9538ce62b11a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:37.033556 29950 tablet_service.cc:1514] Processing DeleteTablet for tablet 011e93d8088a47a2a60db084362c373c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:37 UTC) from {username='oryx', principal='oryx/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:59606
I20260810 00:58:37.033871 30081 tablet_replica.cc:330] T 011e93d8088a47a2a60db084362c373c P 49a07267d0814d14ba4acdf42c845584: stopping tablet replica
I20260810 00:58:37.034101 30081 raft_consensus.cc:2229] T 011e93d8088a47a2a60db084362c373c P 49a07267d0814d14ba4acdf42c845584 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:58:37.034344 30081 raft_consensus.cc:2258] T 011e93d8088a47a2a60db084362c373c P 49a07267d0814d14ba4acdf42c845584 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:37.034678 30082 ts_tablet_manager.cc:1905] T 011e93d8088a47a2a60db084362c373c P 82198c3537184383a48c9538ce62b11a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:37.035920 30081 ts_tablet_manager.cc:1905] T 011e93d8088a47a2a60db084362c373c P 49a07267d0814d14ba4acdf42c845584: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:37.036577 30083 ts_tablet_manager.cc:1905] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:37.041778 30082 ts_tablet_manager.cc:1918] T 011e93d8088a47a2a60db084362c373c P 82198c3537184383a48c9538ce62b11a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:58:37.041949 30082 log.cc:1196] T 011e93d8088a47a2a60db084362c373c P 82198c3537184383a48c9538ce62b11a: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/011e93d8088a47a2a60db084362c373c
I20260810 00:58:37.042852 30081 ts_tablet_manager.cc:1918] T 011e93d8088a47a2a60db084362c373c P 49a07267d0814d14ba4acdf42c845584: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260810 00:58:37.043079 30081 log.cc:1196] T 011e93d8088a47a2a60db084362c373c P 49a07267d0814d14ba4acdf42c845584: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/011e93d8088a47a2a60db084362c373c
I20260810 00:58:37.043576 30081 ts_tablet_manager.cc:1939] T 011e93d8088a47a2a60db084362c373c P 49a07267d0814d14ba4acdf42c845584: Deleting consensus metadata
I20260810 00:58:37.044728 29455 catalog_manager.cc:4841] TS 49a07267d0814d14ba4acdf42c845584 (127.20.81.131:33811): tablet 011e93d8088a47a2a60db084362c373c (table test-table [id=827ff717cd914ec5b02230f09ab52b3c]) successfully deleted
I20260810 00:58:37.052044 30083 ts_tablet_manager.cc:1918] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:58:37.042357 30082 ts_tablet_manager.cc:1939] T 011e93d8088a47a2a60db084362c373c P 82198c3537184383a48c9538ce62b11a: Deleting consensus metadata
I20260810 00:58:37.054536 30083 log.cc:1196] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-0/wal/wals/011e93d8088a47a2a60db084362c373c
I20260810 00:58:37.054989 30083 ts_tablet_manager.cc:1939] T 011e93d8088a47a2a60db084362c373c P 0ac5028bde95499e912b1bb99ba8a7d9: Deleting consensus metadata
I20260810 00:58:37.056025 29457 catalog_manager.cc:4841] TS 82198c3537184383a48c9538ce62b11a (127.20.81.130:45399): tablet 011e93d8088a47a2a60db084362c373c (table test-table [id=827ff717cd914ec5b02230f09ab52b3c]) successfully deleted
I20260810 00:58:37.056171 29455 catalog_manager.cc:4841] TS 0ac5028bde95499e912b1bb99ba8a7d9 (127.20.81.129:45663): tablet 011e93d8088a47a2a60db084362c373c (table test-table [id=827ff717cd914ec5b02230f09ab52b3c]) successfully deleted
Aug 10 00:58:37 dist-test-slave-gp6n krb5kdc[29424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 00:58:37 dist-test-slave-gp6n krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:37 dist-test-slave-gp6n krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 00:58:37.128574 29894 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:37.117681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58182 (local address 127.20.81.190:38311)
0810 00:58:37.117884 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:37.117892 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:37.118251 (+ 359us) server_negotiation.cc:407] Connection header received
0810 00:58:37.118480 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:37.118486 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:37.118580 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:37.118695 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:37.119996 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:37.120773 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:37.122073 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:37.122561 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:37.122701 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:37.128290 (+ 5589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58182: BlockingRecv error: recv got EOF from 127.0.0.1:58182 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 10 00:58:37 dist-test-slave-gp6n krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM
I20260810 00:58:37.147051 29894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:37.131351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58188 (local address 127.20.81.190:38311)
0810 00:58:37.131594 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:37.131599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:37.131936 (+ 337us) server_negotiation.cc:407] Connection header received
0810 00:58:37.132084 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:37.132088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:37.132160 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:37.132240 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:37.133907 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:37.134711 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:37.136574 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:37.136975 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:37.137066 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:37.141639 (+ 4573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:37.141644 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:37.141647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:37.141685 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:37.144418 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:37.145061 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:37.145068 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:37.145072 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:37.145198 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:37.145564 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:37.145570 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:37.145573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:37.145852 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:37.146128 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:37.146464 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 00:58:37.146709 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 00:58:37.150724 29475 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:58:37.151384 29475 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:58:37.159440 29950 tablet_service.cc:1467] Processing CreateTablet for tablet 05819b7bfc33470eb6977344d4db6032 (DEFAULT_TABLE table=test-table [id=823632aa838249d79459681c6663dc53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:37.159471 29786 tablet_service.cc:1467] Processing CreateTablet for tablet 05819b7bfc33470eb6977344d4db6032 (DEFAULT_TABLE table=test-table [id=823632aa838249d79459681c6663dc53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:37.159613 29633 tablet_service.cc:1467] Processing CreateTablet for tablet 05819b7bfc33470eb6977344d4db6032 (DEFAULT_TABLE table=test-table [id=823632aa838249d79459681c6663dc53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:37.159956 29786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05819b7bfc33470eb6977344d4db6032. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:37.159993 29950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05819b7bfc33470eb6977344d4db6032. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:37.160136 29633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05819b7bfc33470eb6977344d4db6032. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:37.166579 30055 tablet_bootstrap.cc:492] T 05819b7bfc33470eb6977344d4db6032 P 82198c3537184383a48c9538ce62b11a: Bootstrap starting.
I20260810 00:58:37.166596 30054 tablet_bootstrap.cc:492] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584: Bootstrap starting.
I20260810 00:58:37.168224 30054 tablet_bootstrap.cc:654] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:37.168426 30055 tablet_bootstrap.cc:654] T 05819b7bfc33470eb6977344d4db6032 P 82198c3537184383a48c9538ce62b11a: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:37.169508 30056 tablet_bootstrap.cc:492] T 05819b7bfc33470eb6977344d4db6032 P 0ac5028bde95499e912b1bb99ba8a7d9: Bootstrap starting.
I20260810 00:58:37.170553 30054 tablet_bootstrap.cc:492] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584: No bootstrap required, opened a new log
I20260810 00:58:37.170730 30054 ts_tablet_manager.cc:1397] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.003s
I20260810 00:58:37.171538 30054 raft_consensus.cc:348] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:37.171809 30054 raft_consensus.cc:374] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:37.171938 30054 raft_consensus.cc:729] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49a07267d0814d14ba4acdf42c845584, State: Initialized, Role: FOLLOWER
I20260810 00:58:37.172145 30056 tablet_bootstrap.cc:654] T 05819b7bfc33470eb6977344d4db6032 P 0ac5028bde95499e912b1bb99ba8a7d9: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:37.172180 30054 consensus_queue.cc:260] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:37.172837 30054 ts_tablet_manager.cc:1428] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 00:58:37.174715 30061 raft_consensus.cc:482] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:58:37.174878 30061 raft_consensus.cc:504] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:37.176018 30056 tablet_bootstrap.cc:492] T 05819b7bfc33470eb6977344d4db6032 P 0ac5028bde95499e912b1bb99ba8a7d9: No bootstrap required, opened a new log
I20260810 00:58:37.176163 30056 ts_tablet_manager.cc:1397] T 05819b7bfc33470eb6977344d4db6032 P 0ac5028bde95499e912b1bb99ba8a7d9: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260810 00:58:37.176719 30061 leader_election.cc:288] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ac5028bde95499e912b1bb99ba8a7d9 (127.20.81.129:45663), 82198c3537184383a48c9538ce62b11a (127.20.81.130:45399)
I20260810 00:58:37.176987 30056 raft_consensus.cc:348] T 05819b7bfc33470eb6977344d4db6032 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:37.177279 30056 raft_consensus.cc:374] T 05819b7bfc33470eb6977344d4db6032 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:37.177448 30056 raft_consensus.cc:729] T 05819b7bfc33470eb6977344d4db6032 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ac5028bde95499e912b1bb99ba8a7d9, State: Initialized, Role: FOLLOWER
I20260810 00:58:37.177773 30056 consensus_queue.cc:260] T 05819b7bfc33470eb6977344d4db6032 P 0ac5028bde95499e912b1bb99ba8a7d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:37.179446 30056 ts_tablet_manager.cc:1428] T 05819b7bfc33470eb6977344d4db6032 P 0ac5028bde95499e912b1bb99ba8a7d9: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260810 00:58:37.196507 30049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:37.177785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:58655 (local address 127.20.81.129:45663)
0810 00:58:37.177997 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:37.178005 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:37.188925 (+ 10920us) server_negotiation.cc:407] Connection header received
0810 00:58:37.189066 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:37.189072 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:37.189158 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:37.189241 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:37.190309 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:37.191513 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:37.194020 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:37.195356 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:37.195504 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:37.195595 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:37.196102 (+ 507us) server_negotiation.cc:299] Negotiation successful
0810 00:58:37.196242 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 00:58:37.197759 30134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:37.177692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:45663 (local address 127.20.81.131:58655)
0810 00:58:37.188819 (+ 11127us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:37.188833 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:37.188957 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:37.189442 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:37.189447 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:37.189475 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:37.190080 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:37.190095 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:37.191743 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:37.191750 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:37.193817 (+ 2067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:37.193831 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:37.195718 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:37.195724 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:37.195903 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:37.195947 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 00:58:37.196866 (+ 919us) client_negotiation.cc:240] Negotiation successful
0810 00:58:37.197025 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11032,"thread_start_us":10122,"threads_started":1}
I20260810 00:58:37.198557 29658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05819b7bfc33470eb6977344d4db6032" candidate_uuid: "49a07267d0814d14ba4acdf42c845584" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" is_pre_election: true
I20260810 00:58:37.198812 29658 raft_consensus.cc:2454] T 05819b7bfc33470eb6977344d4db6032 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49a07267d0814d14ba4acdf42c845584 in term 0.
I20260810 00:58:37.198850 30136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:37.177248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:45399 (local address 127.20.81.131:52779)
0810 00:58:37.188102 (+ 10854us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:37.188129 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:37.188291 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:37.191206 (+ 2915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:37.191210 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:37.191233 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:58:37.191642 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:37.191659 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:37.193389 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:37.193395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:37.195475 (+ 2080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:37.195489 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:37.198037 (+ 2548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:37.198045 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:37.198229 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:37.198263 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 00:58:37.198340 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 00:58:37.198496 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10683,"thread_start_us":11300,"threads_started":1}
I20260810 00:58:37.199026 30055 tablet_bootstrap.cc:492] T 05819b7bfc33470eb6977344d4db6032 P 82198c3537184383a48c9538ce62b11a: No bootstrap required, opened a new log
I20260810 00:58:37.199194 30055 ts_tablet_manager.cc:1397] T 05819b7bfc33470eb6977344d4db6032 P 82198c3537184383a48c9538ce62b11a: Time spent bootstrapping tablet: real 0.033s user 0.006s sys 0.014s
I20260810 00:58:37.199657 29886 leader_election.cc:302] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ac5028bde95499e912b1bb99ba8a7d9, 49a07267d0814d14ba4acdf42c845584; no voters:
I20260810 00:58:37.199883 30135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:37.177318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:52779 (local address 127.20.81.130:45399)
0810 00:58:37.190806 (+ 13488us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:37.190812 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:37.190832 (+ 20us) server_negotiation.cc:407] Connection header received
0810 00:58:37.190897 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:37.190901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:37.191031 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:37.191820 (+ 789us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:58:37.192062 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:37.193233 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:37.195670 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:37.197744 (+ 2074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:37.199240 (+ 1496us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:37.199329 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:37.199384 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 00:58:37.199525 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12525,"thread_start_us":12358,"threads_started":1}
I20260810 00:58:37.200259 30061 raft_consensus.cc:2781] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:58:37.200088 30055 raft_consensus.cc:348] T 05819b7bfc33470eb6977344d4db6032 P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:37.200380 30061 raft_consensus.cc:482] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:58:37.200377 30055 raft_consensus.cc:374] T 05819b7bfc33470eb6977344d4db6032 P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:37.200472 30061 raft_consensus.cc:3037] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:37.200486 29813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05819b7bfc33470eb6977344d4db6032" candidate_uuid: "49a07267d0814d14ba4acdf42c845584" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82198c3537184383a48c9538ce62b11a" is_pre_election: true
I20260810 00:58:37.200497 30055 raft_consensus.cc:729] T 05819b7bfc33470eb6977344d4db6032 P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82198c3537184383a48c9538ce62b11a, State: Initialized, Role: FOLLOWER
I20260810 00:58:37.201125 30055 consensus_queue.cc:260] T 05819b7bfc33470eb6977344d4db6032 P 82198c3537184383a48c9538ce62b11a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:37.201431 29813 raft_consensus.cc:2454] T 05819b7bfc33470eb6977344d4db6032 P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49a07267d0814d14ba4acdf42c845584 in term 0.
I20260810 00:58:37.202548 30055 ts_tablet_manager.cc:1428] T 05819b7bfc33470eb6977344d4db6032 P 82198c3537184383a48c9538ce62b11a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 00:58:37.203218 30061 raft_consensus.cc:504] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:37.203792 30061 leader_election.cc:288] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584 [CANDIDATE]: Term 1 election: Requested vote from peers 0ac5028bde95499e912b1bb99ba8a7d9 (127.20.81.129:45663), 82198c3537184383a48c9538ce62b11a (127.20.81.130:45399)
I20260810 00:58:37.204241 29813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05819b7bfc33470eb6977344d4db6032" candidate_uuid: "49a07267d0814d14ba4acdf42c845584" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82198c3537184383a48c9538ce62b11a"
I20260810 00:58:37.204547 29813 raft_consensus.cc:3037] T 05819b7bfc33470eb6977344d4db6032 P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:37.205118 29658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05819b7bfc33470eb6977344d4db6032" candidate_uuid: "49a07267d0814d14ba4acdf42c845584" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ac5028bde95499e912b1bb99ba8a7d9"
I20260810 00:58:37.205367 29658 raft_consensus.cc:3037] T 05819b7bfc33470eb6977344d4db6032 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:37.207878 29813 raft_consensus.cc:2454] T 05819b7bfc33470eb6977344d4db6032 P 82198c3537184383a48c9538ce62b11a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49a07267d0814d14ba4acdf42c845584 in term 1.
I20260810 00:58:37.208106 29658 raft_consensus.cc:2454] T 05819b7bfc33470eb6977344d4db6032 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49a07267d0814d14ba4acdf42c845584 in term 1.
I20260810 00:58:37.208349 29888 leader_election.cc:302] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49a07267d0814d14ba4acdf42c845584, 82198c3537184383a48c9538ce62b11a; no voters:
I20260810 00:58:37.210057 30061 raft_consensus.cc:2781] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:58:37.210649 30061 raft_consensus.cc:686] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584 [term 1 LEADER]: Becoming Leader. State: Replica: 49a07267d0814d14ba4acdf42c845584, State: Running, Role: LEADER
I20260810 00:58:37.211274 30061 consensus_queue.cc:237] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:37.215096 29476 catalog_manager.cc:5495] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49a07267d0814d14ba4acdf42c845584 (127.20.81.131). New cstate: current_term: 1 leader_uuid: "49a07267d0814d14ba4acdf42c845584" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } health_report { overall_health: HEALTHY } } }
I20260810 00:58:37.231443 30140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:37.224742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59608 (local address 127.20.81.131:33811)
0810 00:58:37.225336 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:37.225340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:37.225357 (+ 17us) server_negotiation.cc:407] Connection header received
0810 00:58:37.225435 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:37.225439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:37.225505 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:37.225670 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:58:37.226096 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:37.226979 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:37.228472 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:37.229249 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:37.229411 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:37.229862 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:58:37.230697 (+ 835us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:58:37.230855 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:37.230933 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 00:58:37.231025 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":326,"threads_started":1}
I20260810 00:58:37.235476 29658 raft_consensus.cc:1261] T 05819b7bfc33470eb6977344d4db6032 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 1 FOLLOWER]: Refusing update from remote peer 49a07267d0814d14ba4acdf42c845584: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:58:37.235637 29813 raft_consensus.cc:1261] T 05819b7bfc33470eb6977344d4db6032 P 82198c3537184383a48c9538ce62b11a [term 1 FOLLOWER]: Refusing update from remote peer 49a07267d0814d14ba4acdf42c845584: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:58:37.236295 30137 consensus_queue.cc:1035] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:58:37.236650 30061 consensus_queue.cc:1035] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:58:37.259850 29475 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:58:37.260160 29475 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:58:37.264330 29475 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4: Sending DeleteTablet for 3 replicas of tablet 05819b7bfc33470eb6977344d4db6032
I20260810 00:58:37.265594 29633 tablet_service.cc:1514] Processing DeleteTablet for tablet 05819b7bfc33470eb6977344d4db6032 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:37 UTC) from {username='oryx', principal='oryx/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:46750
I20260810 00:58:37.265851 29950 tablet_service.cc:1514] Processing DeleteTablet for tablet 05819b7bfc33470eb6977344d4db6032 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:37 UTC) from {username='oryx', principal='oryx/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:59606
I20260810 00:58:37.265941 30083 tablet_replica.cc:330] T 05819b7bfc33470eb6977344d4db6032 P 0ac5028bde95499e912b1bb99ba8a7d9: stopping tablet replica
I20260810 00:58:37.266152 30083 raft_consensus.cc:2229] T 05819b7bfc33470eb6977344d4db6032 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:58:37.266455 30083 raft_consensus.cc:2258] T 05819b7bfc33470eb6977344d4db6032 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:37.266613 30081 tablet_replica.cc:330] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584: stopping tablet replica
I20260810 00:58:37.266839 30081 raft_consensus.cc:2229] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:58:37.267442 30081 raft_consensus.cc:2258] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:37.268040 30083 ts_tablet_manager.cc:1905] T 05819b7bfc33470eb6977344d4db6032 P 0ac5028bde95499e912b1bb99ba8a7d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:37.268020 29786 tablet_service.cc:1514] Processing DeleteTablet for tablet 05819b7bfc33470eb6977344d4db6032 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:37 UTC) from {username='oryx', principal='oryx/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:50564
I20260810 00:58:37.268392 30082 tablet_replica.cc:330] T 05819b7bfc33470eb6977344d4db6032 P 82198c3537184383a48c9538ce62b11a: stopping tablet replica
I20260810 00:58:37.268600 30081 ts_tablet_manager.cc:1905] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:37.268599 30082 raft_consensus.cc:2229] T 05819b7bfc33470eb6977344d4db6032 P 82198c3537184383a48c9538ce62b11a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:58:37.268869 30082 raft_consensus.cc:2258] T 05819b7bfc33470eb6977344d4db6032 P 82198c3537184383a48c9538ce62b11a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:37.270615 29475 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:58:37.271248 29475 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:58:37.273248 30082 ts_tablet_manager.cc:1905] T 05819b7bfc33470eb6977344d4db6032 P 82198c3537184383a48c9538ce62b11a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:37.277359 30083 ts_tablet_manager.cc:1918] T 05819b7bfc33470eb6977344d4db6032 P 0ac5028bde95499e912b1bb99ba8a7d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:58:37.277609 30083 log.cc:1196] T 05819b7bfc33470eb6977344d4db6032 P 0ac5028bde95499e912b1bb99ba8a7d9: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-0/wal/wals/05819b7bfc33470eb6977344d4db6032
I20260810 00:58:37.278174 30083 ts_tablet_manager.cc:1939] T 05819b7bfc33470eb6977344d4db6032 P 0ac5028bde95499e912b1bb99ba8a7d9: Deleting consensus metadata
I20260810 00:58:37.280062 29950 tablet_service.cc:1467] Processing CreateTablet for tablet a363b905be8847d284590631e7321187 (DEFAULT_TABLE table=test-table [id=86d9acfa1f114001afaf66856c9d8fc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:37.280493 29950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a363b905be8847d284590631e7321187. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:37.281184 30082 ts_tablet_manager.cc:1918] T 05819b7bfc33470eb6977344d4db6032 P 82198c3537184383a48c9538ce62b11a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:58:37.281324 30082 log.cc:1196] T 05819b7bfc33470eb6977344d4db6032 P 82198c3537184383a48c9538ce62b11a: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/05819b7bfc33470eb6977344d4db6032
I20260810 00:58:37.281476 30081 ts_tablet_manager.cc:1918] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:58:37.281636 30081 log.cc:1196] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/05819b7bfc33470eb6977344d4db6032
I20260810 00:58:37.281754 30082 ts_tablet_manager.cc:1939] T 05819b7bfc33470eb6977344d4db6032 P 82198c3537184383a48c9538ce62b11a: Deleting consensus metadata
I20260810 00:58:37.282123 30081 ts_tablet_manager.cc:1939] T 05819b7bfc33470eb6977344d4db6032 P 49a07267d0814d14ba4acdf42c845584: Deleting consensus metadata
I20260810 00:58:37.283254 29457 catalog_manager.cc:4841] TS 82198c3537184383a48c9538ce62b11a (127.20.81.130:45399): tablet 05819b7bfc33470eb6977344d4db6032 (table test-table [id=823632aa838249d79459681c6663dc53]) successfully deleted
I20260810 00:58:37.283798 29786 tablet_service.cc:1467] Processing CreateTablet for tablet a363b905be8847d284590631e7321187 (DEFAULT_TABLE table=test-table [id=86d9acfa1f114001afaf66856c9d8fc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:37.283939 29455 catalog_manager.cc:4841] TS 49a07267d0814d14ba4acdf42c845584 (127.20.81.131:33811): tablet 05819b7bfc33470eb6977344d4db6032 (table test-table [id=823632aa838249d79459681c6663dc53]) successfully deleted
I20260810 00:58:37.284219 29786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a363b905be8847d284590631e7321187. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:37.288785 30054 tablet_bootstrap.cc:492] T a363b905be8847d284590631e7321187 P 49a07267d0814d14ba4acdf42c845584: Bootstrap starting.
I20260810 00:58:37.283331 29633 tablet_service.cc:1467] Processing CreateTablet for tablet a363b905be8847d284590631e7321187 (DEFAULT_TABLE table=test-table [id=86d9acfa1f114001afaf66856c9d8fc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:58:37.289371 29633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a363b905be8847d284590631e7321187. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:37.290232 30055 tablet_bootstrap.cc:492] T a363b905be8847d284590631e7321187 P 82198c3537184383a48c9538ce62b11a: Bootstrap starting.
I20260810 00:58:37.290561 30054 tablet_bootstrap.cc:654] T a363b905be8847d284590631e7321187 P 49a07267d0814d14ba4acdf42c845584: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:37.291910 30055 tablet_bootstrap.cc:654] T a363b905be8847d284590631e7321187 P 82198c3537184383a48c9538ce62b11a: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:37.297751 30056 tablet_bootstrap.cc:492] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9: Bootstrap starting.
I20260810 00:58:37.302361 30055 tablet_bootstrap.cc:492] T a363b905be8847d284590631e7321187 P 82198c3537184383a48c9538ce62b11a: No bootstrap required, opened a new log
I20260810 00:58:37.302702 30055 ts_tablet_manager.cc:1397] T a363b905be8847d284590631e7321187 P 82198c3537184383a48c9538ce62b11a: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.005s
I20260810 00:58:37.302711 30056 tablet_bootstrap.cc:654] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:37.303562 30055 raft_consensus.cc:348] T a363b905be8847d284590631e7321187 P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:37.303905 30055 raft_consensus.cc:374] T a363b905be8847d284590631e7321187 P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:37.304034 30055 raft_consensus.cc:729] T a363b905be8847d284590631e7321187 P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82198c3537184383a48c9538ce62b11a, State: Initialized, Role: FOLLOWER
I20260810 00:58:37.304385 30055 consensus_queue.cc:260] T a363b905be8847d284590631e7321187 P 82198c3537184383a48c9538ce62b11a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:37.306706 30056 tablet_bootstrap.cc:492] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9: No bootstrap required, opened a new log
I20260810 00:58:37.306905 30056 ts_tablet_manager.cc:1397] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260810 00:58:37.307564 30055 ts_tablet_manager.cc:1428] T a363b905be8847d284590631e7321187 P 82198c3537184383a48c9538ce62b11a: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260810 00:58:37.307744 30056 raft_consensus.cc:348] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:37.308022 30056 raft_consensus.cc:374] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:37.308112 30056 raft_consensus.cc:729] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ac5028bde95499e912b1bb99ba8a7d9, State: Initialized, Role: FOLLOWER
I20260810 00:58:37.308460 30056 consensus_queue.cc:260] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:37.310034 29455 catalog_manager.cc:4841] TS 0ac5028bde95499e912b1bb99ba8a7d9 (127.20.81.129:45663): tablet 05819b7bfc33470eb6977344d4db6032 (table test-table [id=823632aa838249d79459681c6663dc53]) successfully deleted
I20260810 00:58:37.310887 30056 ts_tablet_manager.cc:1428] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260810 00:58:37.313254 30054 tablet_bootstrap.cc:492] T a363b905be8847d284590631e7321187 P 49a07267d0814d14ba4acdf42c845584: No bootstrap required, opened a new log
I20260810 00:58:37.313468 30054 ts_tablet_manager.cc:1397] T a363b905be8847d284590631e7321187 P 49a07267d0814d14ba4acdf42c845584: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.010s
I20260810 00:58:37.314445 30054 raft_consensus.cc:348] T a363b905be8847d284590631e7321187 P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:37.314733 30054 raft_consensus.cc:374] T a363b905be8847d284590631e7321187 P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:37.314826 30054 raft_consensus.cc:729] T a363b905be8847d284590631e7321187 P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49a07267d0814d14ba4acdf42c845584, State: Initialized, Role: FOLLOWER
I20260810 00:58:37.315104 30054 consensus_queue.cc:260] T a363b905be8847d284590631e7321187 P 49a07267d0814d14ba4acdf42c845584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:37.316843 30054 ts_tablet_manager.cc:1428] T a363b905be8847d284590631e7321187 P 49a07267d0814d14ba4acdf42c845584: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 00:58:37.398974 30115 raft_consensus.cc:482] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:58:37.399207 30115 raft_consensus.cc:504] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:37.399940 30115 leader_election.cc:288] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82198c3537184383a48c9538ce62b11a (127.20.81.130:45399), 49a07267d0814d14ba4acdf42c845584 (127.20.81.131:33811)
I20260810 00:58:37.400585 29813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a363b905be8847d284590631e7321187" candidate_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82198c3537184383a48c9538ce62b11a" is_pre_election: true
I20260810 00:58:37.400730 29974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a363b905be8847d284590631e7321187" candidate_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49a07267d0814d14ba4acdf42c845584" is_pre_election: true
I20260810 00:58:37.400887 29813 raft_consensus.cc:2454] T a363b905be8847d284590631e7321187 P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ac5028bde95499e912b1bb99ba8a7d9 in term 0.
I20260810 00:58:37.401017 29974 raft_consensus.cc:2454] T a363b905be8847d284590631e7321187 P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ac5028bde95499e912b1bb99ba8a7d9 in term 0.
I20260810 00:58:37.401571 29574 leader_election.cc:302] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ac5028bde95499e912b1bb99ba8a7d9, 49a07267d0814d14ba4acdf42c845584; no voters:
I20260810 00:58:37.401898 30115 raft_consensus.cc:2781] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:58:37.402011 30115 raft_consensus.cc:482] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:58:37.402087 30115 raft_consensus.cc:3037] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:37.405267 30115 raft_consensus.cc:504] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:37.406380 29813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a363b905be8847d284590631e7321187" candidate_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82198c3537184383a48c9538ce62b11a"
I20260810 00:58:37.406435 29974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a363b905be8847d284590631e7321187" candidate_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49a07267d0814d14ba4acdf42c845584"
I20260810 00:58:37.406647 29813 raft_consensus.cc:3037] T a363b905be8847d284590631e7321187 P 82198c3537184383a48c9538ce62b11a [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:37.406678 29974 raft_consensus.cc:3037] T a363b905be8847d284590631e7321187 P 49a07267d0814d14ba4acdf42c845584 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:37.409925 29974 raft_consensus.cc:2454] T a363b905be8847d284590631e7321187 P 49a07267d0814d14ba4acdf42c845584 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ac5028bde95499e912b1bb99ba8a7d9 in term 1.
I20260810 00:58:37.410493 29813 raft_consensus.cc:2454] T a363b905be8847d284590631e7321187 P 82198c3537184383a48c9538ce62b11a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ac5028bde95499e912b1bb99ba8a7d9 in term 1.
I20260810 00:58:37.410506 29574 leader_election.cc:302] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ac5028bde95499e912b1bb99ba8a7d9, 49a07267d0814d14ba4acdf42c845584; no voters:
I20260810 00:58:37.410846 30063 raft_consensus.cc:2781] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:58:37.411011 30063 raft_consensus.cc:686] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 1 LEADER]: Becoming Leader. State: Replica: 0ac5028bde95499e912b1bb99ba8a7d9, State: Running, Role: LEADER
I20260810 00:58:37.411281 30063 consensus_queue.cc:237] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } }
I20260810 00:58:37.410511 30115 leader_election.cc:288] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9 [CANDIDATE]: Term 1 election: Requested vote from peers 82198c3537184383a48c9538ce62b11a (127.20.81.130:45399), 49a07267d0814d14ba4acdf42c845584 (127.20.81.131:33811)
I20260810 00:58:37.414465 29475 catalog_manager.cc:5495] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ac5028bde95499e912b1bb99ba8a7d9 (127.20.81.129). New cstate: current_term: 1 leader_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac5028bde95499e912b1bb99ba8a7d9" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 } health_report { overall_health: UNKNOWN } } }
I20260810 00:58:37.469915 30049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:37.463002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46792 (local address 127.20.81.129:45663)
0810 00:58:37.463655 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:37.463662 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:37.463679 (+ 17us) server_negotiation.cc:407] Connection header received
0810 00:58:37.463740 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:37.463745 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:37.463834 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:37.463940 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:58:37.464364 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:37.465592 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:37.467658 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:37.468370 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:37.468505 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:37.469053 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:58:37.469218 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:58:37.469340 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:37.469489 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 00:58:37.469572 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511}
I20260810 00:58:37.484650 29974 raft_consensus.cc:1261] T a363b905be8847d284590631e7321187 P 49a07267d0814d14ba4acdf42c845584 [term 1 FOLLOWER]: Refusing update from remote peer 0ac5028bde95499e912b1bb99ba8a7d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:58:37.484669 29813 raft_consensus.cc:1261] T a363b905be8847d284590631e7321187 P 82198c3537184383a48c9538ce62b11a [term 1 FOLLOWER]: Refusing update from remote peer 0ac5028bde95499e912b1bb99ba8a7d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:58:37.485384 30091 consensus_queue.cc:1035] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82198c3537184383a48c9538ce62b11a" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 45399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:58:37.485765 30063 consensus_queue.cc:1035] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49a07267d0814d14ba4acdf42c845584" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:58:37.599907 29474 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:58:37.600200 29474 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:58:37.604501 29474 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5dad884925fc477b833f1b2a578b46e4: Sending DeleteTablet for 3 replicas of tablet a363b905be8847d284590631e7321187
I20260810 00:58:37.605718 29786 tablet_service.cc:1514] Processing DeleteTablet for tablet a363b905be8847d284590631e7321187 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:37 UTC) from {username='oryx', principal='oryx/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:50564
I20260810 00:58:37.606034 30082 tablet_replica.cc:330] T a363b905be8847d284590631e7321187 P 82198c3537184383a48c9538ce62b11a: stopping tablet replica
I20260810 00:58:37.606084 29633 tablet_service.cc:1514] Processing DeleteTablet for tablet a363b905be8847d284590631e7321187 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:37 UTC) from {username='oryx', principal='oryx/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:46750
I20260810 00:58:37.606251 30082 raft_consensus.cc:2229] T a363b905be8847d284590631e7321187 P 82198c3537184383a48c9538ce62b11a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:58:37.606460 30083 tablet_replica.cc:330] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9: stopping tablet replica
I20260810 00:58:37.606472 30082 raft_consensus.cc:2258] T a363b905be8847d284590631e7321187 P 82198c3537184383a48c9538ce62b11a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:37.606693 30083 raft_consensus.cc:2229] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:58:37.607070 30083 raft_consensus.cc:2258] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:37.608111 29950 tablet_service.cc:1514] Processing DeleteTablet for tablet a363b905be8847d284590631e7321187 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:58:37 UTC) from {username='oryx', principal='oryx/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:59606
I20260810 00:58:37.608487 30081 tablet_replica.cc:330] T a363b905be8847d284590631e7321187 P 49a07267d0814d14ba4acdf42c845584: stopping tablet replica
I20260810 00:58:37.608641 30082 ts_tablet_manager.cc:1905] T a363b905be8847d284590631e7321187 P 82198c3537184383a48c9538ce62b11a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:37.608718 30081 raft_consensus.cc:2229] T a363b905be8847d284590631e7321187 P 49a07267d0814d14ba4acdf42c845584 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:58:37.608953 30081 raft_consensus.cc:2258] T a363b905be8847d284590631e7321187 P 49a07267d0814d14ba4acdf42c845584 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:58:37.610558 30081 ts_tablet_manager.cc:1905] T a363b905be8847d284590631e7321187 P 49a07267d0814d14ba4acdf42c845584: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:37.613260 30083 ts_tablet_manager.cc:1905] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:58:37.615684 30082 ts_tablet_manager.cc:1918] T a363b905be8847d284590631e7321187 P 82198c3537184383a48c9538ce62b11a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:58:37.615864 30082 log.cc:1196] T a363b905be8847d284590631e7321187 P 82198c3537184383a48c9538ce62b11a: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/a363b905be8847d284590631e7321187
I20260810 00:58:37.616298 30082 ts_tablet_manager.cc:1939] T a363b905be8847d284590631e7321187 P 82198c3537184383a48c9538ce62b11a: Deleting consensus metadata
I20260810 00:58:37.617475 29457 catalog_manager.cc:4841] TS 82198c3537184383a48c9538ce62b11a (127.20.81.130:45399): tablet a363b905be8847d284590631e7321187 (table test-table [id=86d9acfa1f114001afaf66856c9d8fc3]) successfully deleted
I20260810 00:58:37.618831 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 29557
I20260810 00:58:37.627040 30083 ts_tablet_manager.cc:1918] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:58:37.627336 30083 log.cc:1196] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-0/wal/wals/a363b905be8847d284590631e7321187
I20260810 00:58:37.627931 30083 ts_tablet_manager.cc:1939] T a363b905be8847d284590631e7321187 P 0ac5028bde95499e912b1bb99ba8a7d9: Deleting consensus metadata
I20260810 00:58:37.629312 29455 catalog_manager.cc:4841] TS 0ac5028bde95499e912b1bb99ba8a7d9 (127.20.81.129:45663): tablet a363b905be8847d284590631e7321187 (table test-table [id=86d9acfa1f114001afaf66856c9d8fc3]) successfully deleted
I20260810 00:58:37.631911 30081 ts_tablet_manager.cc:1918] T a363b905be8847d284590631e7321187 P 49a07267d0814d14ba4acdf42c845584: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:58:37.632323 30081 log.cc:1196] T a363b905be8847d284590631e7321187 P 49a07267d0814d14ba4acdf42c845584: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/a363b905be8847d284590631e7321187
I20260810 00:58:37.632823 30081 ts_tablet_manager.cc:1939] T a363b905be8847d284590631e7321187 P 49a07267d0814d14ba4acdf42c845584: Deleting consensus metadata
I20260810 00:58:37.634299 29455 catalog_manager.cc:4841] TS 49a07267d0814d14ba4acdf42c845584 (127.20.81.131:33811): tablet a363b905be8847d284590631e7321187 (table test-table [id=86d9acfa1f114001afaf66856c9d8fc3]) successfully deleted
Aug 10 00:58:37 dist-test-slave-gp6n krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.81.129@KRBTEST.COM
I20260810 00:58:37.652086 30049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:37.623028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46808 (local address 127.20.81.129:45663)
0810 00:58:37.623200 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:37.623206 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:37.630264 (+ 7058us) server_negotiation.cc:407] Connection header received
0810 00:58:37.630659 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:37.630665 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:37.630765 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:37.630865 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:37.632860 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:37.634071 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:37.636540 (+ 2469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:37.637223 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:37.637365 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:37.643364 (+ 5999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:37.643392 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:37.643415 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:37.643463 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:37.648412 (+ 4949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:37.649373 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:37.649385 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:37.649388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:37.649583 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:37.650074 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:37.650079 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:37.650083 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:37.650659 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:37.651078 (+ 419us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:37.651359 (+ 281us) server_negotiation.cc:299] Negotiation successful
0810 00:58:37.651632 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 00:58:37.652886 29704 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:37.802820 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 29557
I20260810 00:58:37.835696 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 29714
Aug 10 00:58:37 dist-test-slave-gp6n krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.81.130@KRBTEST.COM
I20260810 00:58:37.851104 30156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:37.836541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50590 (local address 127.20.81.130:45399)
0810 00:58:37.837025 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:37.837030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:37.837045 (+ 15us) server_negotiation.cc:407] Connection header received
0810 00:58:37.837100 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:37.837103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:37.837160 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:37.837279 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:37.838641 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:37.839368 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:37.840671 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:37.841079 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:37.841222 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:37.845367 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:37.845381 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:37.845398 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:37.845436 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:37.848342 (+ 2906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:37.848984 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:37.848993 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:37.848995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:37.849120 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:37.849513 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:37.849519 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:37.849522 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:37.849876 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:37.850210 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:37.850526 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 00:58:37.850755 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":225,"threads_started":1}
I20260810 00:58:37.851738 29859 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:37.976986 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 29714
I20260810 00:58:38.002663 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 29869
Aug 10 00:58:38 dist-test-slave-gp6n krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.81.131@KRBTEST.COM
I20260810 00:58:38.018743 30158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:38.003618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59620 (local address 127.20.81.131:33811)
0810 00:58:38.004071 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:38.004075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:38.004093 (+ 18us) server_negotiation.cc:407] Connection header received
0810 00:58:38.004145 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:38.004149 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:38.004209 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:38.004300 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:38.005698 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:38.006538 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:38.007757 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:38.008184 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:38.008294 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:38.013241 (+ 4947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:38.013269 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:38.013286 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:38.013366 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:38.016281 (+ 2915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:38.016898 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:38.016905 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:38.016907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:38.017063 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:38.017387 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:38.017390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:38.017392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:38.017732 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:38.017978 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:38.018195 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 00:58:38.018357 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":257,"threads_started":1}
I20260810 00:58:38.019428 30020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:38.144462 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 29869
I20260810 00:58:38.169163 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 29439
I20260810 00:58:38.170645 29524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:38.295506 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 29439
2026-08-10T00:58:38Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6877 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:58:38 dist-test-slave-gp6n krb5kdc[30164](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:58:38 dist-test-slave-gp6n krb5kdc[30164](info): set up 2 sockets
Aug 10 00:58:38 dist-test-slave-gp6n krb5kdc[30164](info): commencing operation
krb5kdc: starting...
W20260810 00:58:39.503918 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 00:58:39 dist-test-slave-gp6n krb5kdc[30164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:58:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:58:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 00:58:39.771093 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:35431
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:46521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:35431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:40.108964 30179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:40.109390 30179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:40.109575 30179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:40.117419 30179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:58:40.117574 30179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:40.117686 30179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:40.117791 30179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:58:40.117888 30179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:58:40.127921 30179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:46521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:35431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:35431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:40.129251 30179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:40.130716 30179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:40.143368 30186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:40.143399 30184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:40.143718 30179 server_base.cc:1034] running on GCE node
W20260810 00:58:40.143970 30188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:40.144832 30179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:40.146281 30179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:40.147534 30179 hybrid_clock.cc:648] HybridClock initialized: now 1786323520147481 us; error 41 us; skew 500 ppm
I20260810 00:58:40.147840 30179 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T00:58:40Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1918 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:58:40 dist-test-slave-gp6n krb5kdc[30196](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:58:40 dist-test-slave-gp6n krb5kdc[30196](info): set up 2 sockets
Aug 10 00:58:40 dist-test-slave-gp6n krb5kdc[30196](info): commencing operation
krb5kdc: starting...
W20260810 00:58:41.396733 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 00:58:41 dist-test-slave-gp6n krb5kdc[30196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:58:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:58:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 00:58:41.662557 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:44873
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:44873
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:42.001214 30210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:42.001499 30210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:42.001590 30210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:42.008693 30210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:58:42.008808 30210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:42.008857 30210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:42.008914 30210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:58:42.008960 30210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:58:42.018846 30210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:44873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:44873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:42.020145 30210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:42.021473 30210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:42.033718 30215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:42.033607 30217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:42.033818 30220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:42.034466 30210 server_base.cc:1034] running on GCE node
I20260810 00:58:42.035303 30210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:42.036763 30210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:42.037968 30210 hybrid_clock.cc:648] HybridClock initialized: now 1786323522037911 us; error 52 us; skew 500 ppm
I20260810 00:58:42.038306 30210 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:42 dist-test-slave-gp6n krb5kdc[30196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323522, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:42.045961 30210 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 00:58:42.047482 30210 webserver.cc:460] Webserver started at http://127.20.81.190:45935/ using document root <none> and password file <none>
I20260810 00:58:42.048152 30210 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:42.048257 30210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:42.048525 30210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:42.051566 30210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "587389e4d023465e94ded04f4a6ca8e1"
format_stamp: "Formatted at 2026-08-10 00:58:42 on dist-test-slave-gp6n"
server_key: "fd820f0bd46c07ebdc51772f13dfc158"
server_key_iv: "0e2b28c86252b16cd1677aa1c2c1611f"
server_key_version: "encryptionkey@0"
I20260810 00:58:42.052297 30210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "587389e4d023465e94ded04f4a6ca8e1"
format_stamp: "Formatted at 2026-08-10 00:58:42 on dist-test-slave-gp6n"
server_key: "fd820f0bd46c07ebdc51772f13dfc158"
server_key_iv: "0e2b28c86252b16cd1677aa1c2c1611f"
server_key_version: "encryptionkey@0"
I20260810 00:58:42.057811 30210 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 00:58:42.061233 30229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:42.062808 30210 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 00:58:42.063006 30210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "587389e4d023465e94ded04f4a6ca8e1"
format_stamp: "Formatted at 2026-08-10 00:58:42 on dist-test-slave-gp6n"
server_key: "fd820f0bd46c07ebdc51772f13dfc158"
server_key_iv: "0e2b28c86252b16cd1677aa1c2c1611f"
server_key_version: "encryptionkey@0"
I20260810 00:58:42.063218 30210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:42.087106 30210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:42.091460 30210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:42.091784 30210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:42.112439 30210 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:44873
I20260810 00:58:42.112478 30284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:44873 every 8 connection(s)
I20260810 00:58:42.113967 30210 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:58:42.117946 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 30210
I20260810 00:58:42.118244 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 00:58:42.118495 30285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:58:42.119205 20806 external_mini_cluster.cc:1428] Setting key d7a82521fe462dc1f67b5d0539f5eb72
I20260810 00:58:42.135141 30285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1: Bootstrap starting.
Aug 10 00:58:42 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:58:42.139213 30285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1: Neither blocks nor log segments found. Creating new log.
I20260810 00:58:42.140597 30285 log.cc:824] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:42.144102 30285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1: No bootstrap required, opened a new log
I20260810 00:58:42.145351 30288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:42.121512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57748 (local address 127.20.81.190:44873)
0810 00:58:42.122331 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:42.122346 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:42.122395 (+ 49us) server_negotiation.cc:407] Connection header received
0810 00:58:42.124266 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:42.124289 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:42.124686 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:42.125007 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:42.127127 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:42.128532 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:42.130534 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:42.131245 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:42.131454 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:42.136559 (+ 5105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:42.136571 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:42.136590 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:42.136647 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:42.140744 (+ 4097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:42.141624 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:42.141630 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:42.141636 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:42.141813 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:42.142195 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:42.142199 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:42.142202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:42.142887 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:42.143201 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:42.143573 (+ 372us) server_negotiation.cc:299] Negotiation successful
0810 00:58:42.144003 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":412,"threads_started":1}
I20260810 00:58:42.152343 30285 raft_consensus.cc:348] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587389e4d023465e94ded04f4a6ca8e1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44873 } }
I20260810 00:58:42.152667 30285 raft_consensus.cc:374] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:58:42.152745 30285 raft_consensus.cc:729] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 587389e4d023465e94ded04f4a6ca8e1, State: Initialized, Role: FOLLOWER
I20260810 00:58:42.153272 30285 consensus_queue.cc:260] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587389e4d023465e94ded04f4a6ca8e1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44873 } }
I20260810 00:58:42.153488 30285 raft_consensus.cc:388] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:58:42.153620 30285 raft_consensus.cc:482] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:58:42.153797 30285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:58:42.156144 30285 raft_consensus.cc:504] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587389e4d023465e94ded04f4a6ca8e1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44873 } }
I20260810 00:58:42.156565 30285 leader_election.cc:302] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 587389e4d023465e94ded04f4a6ca8e1; no voters:
I20260810 00:58:42.157150 30285 leader_election.cc:288] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:58:42.157228 30290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:58:42.158285 30290 raft_consensus.cc:686] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [term 1 LEADER]: Becoming Leader. State: Replica: 587389e4d023465e94ded04f4a6ca8e1, State: Running, Role: LEADER
I20260810 00:58:42.158774 30290 consensus_queue.cc:237] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587389e4d023465e94ded04f4a6ca8e1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44873 } }
I20260810 00:58:42.159291 30285 sys_catalog.cc:564] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:58:42.161368 30294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "587389e4d023465e94ded04f4a6ca8e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587389e4d023465e94ded04f4a6ca8e1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44873 } } }
I20260810 00:58:42.161738 30294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:42.162313 30295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 587389e4d023465e94ded04f4a6ca8e1. Latest consensus state: current_term: 1 leader_uuid: "587389e4d023465e94ded04f4a6ca8e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587389e4d023465e94ded04f4a6ca8e1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44873 } } }
I20260810 00:58:42.162859 30295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:42.163739 30299 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:58:42.168814 30299 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:58:42.177685 30299 catalog_manager.cc:1347] Generated new cluster ID: f11869958bc44e58bcb397809ac63495
I20260810 00:58:42.177812 30299 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:58:42.187639 30299 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:58:42.188884 30299 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:58:42.195537 30299 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1: Generated new TSK 0
I20260810 00:58:42.196254 30299 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.81.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
I20260810 00:58:42.274756 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44873
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:42.620810 30314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:42.621083 30314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:42.621249 30314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:42.628611 30314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:42.628711 30314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:42.628894 30314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 00:58:42.638299 30314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:42.639586 30314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:42.640859 30314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:42.655529 30320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:42.655615 30322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:42.655611 30319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:42.656478 30314 server_base.cc:1034] running on GCE node
I20260810 00:58:42.657038 30314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:42.658231 30314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:42.659509 30314 hybrid_clock.cc:648] HybridClock initialized: now 1786323522659413 us; error 80 us; skew 500 ppm
I20260810 00:58:42.659866 30314 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:42 dist-test-slave-gp6n krb5kdc[30196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323522, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:42.664155 30314 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 00:58:42.665774 30314 webserver.cc:460] Webserver started at http://127.20.81.129:37855/ using document root <none> and password file <none>
I20260810 00:58:42.666543 30314 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:42.666683 30314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:42.667047 30314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:42.670125 30314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "080086e2c5c8414d948a348684d713b0"
format_stamp: "Formatted at 2026-08-10 00:58:42 on dist-test-slave-gp6n"
server_key: "7083e52eac8f5e77f23be70e88d52bc8"
server_key_iv: "970ca24faafcc0c69ce1888e59b048e6"
server_key_version: "encryptionkey@0"
I20260810 00:58:42.670994 30314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "080086e2c5c8414d948a348684d713b0"
format_stamp: "Formatted at 2026-08-10 00:58:42 on dist-test-slave-gp6n"
server_key: "7083e52eac8f5e77f23be70e88d52bc8"
server_key_iv: "970ca24faafcc0c69ce1888e59b048e6"
server_key_version: "encryptionkey@0"
I20260810 00:58:42.676378 30314 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 00:58:42.679883 30329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:42.681228 30314 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 00:58:42.681519 30314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "080086e2c5c8414d948a348684d713b0"
format_stamp: "Formatted at 2026-08-10 00:58:42 on dist-test-slave-gp6n"
server_key: "7083e52eac8f5e77f23be70e88d52bc8"
server_key_iv: "970ca24faafcc0c69ce1888e59b048e6"
server_key_version: "encryptionkey@0"
I20260810 00:58:42.681730 30314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:42.711467 30314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:42.715888 30314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:42.716187 30314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:42.717166 30314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:42.718585 30314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:42.718683 30314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:42.718832 30314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:42.718912 30314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:42.752671 30314 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:35173
I20260810 00:58:42.752761 30455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:35173 every 8 connection(s)
I20260810 00:58:42.754032 30314 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 00:58:42.754789 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 30314
I20260810 00:58:42.755057 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 00:58:42.756013 20806 external_mini_cluster.cc:1428] Setting key 5aa9cf0486a5745dd811cd24a2ff01e2
Aug 10 00:58:42 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323522, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:58:42.781301 30459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:42.757298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:36331 (local address 127.20.81.190:44873)
0810 00:58:42.757851 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:42.757858 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:42.759010 (+ 1152us) server_negotiation.cc:407] Connection header received
0810 00:58:42.762174 (+ 3164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:42.762178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:42.762252 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:42.762363 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:42.764747 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:42.765770 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:42.768606 (+ 2836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:42.769063 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:42.769174 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:42.773519 (+ 4345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:42.773528 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:42.773536 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:42.773584 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:42.777394 (+ 3810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:42.778277 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:42.778286 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:42.778292 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:42.778486 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:42.779032 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:42.779044 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:42.779050 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:42.779478 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:42.779645 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:42.780509 (+ 864us) server_negotiation.cc:299] Negotiation successful
0810 00:58:42.780805 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":260,"threads_started":1}
I20260810 00:58:42.782051 30460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:42.757915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.129:36331)
0810 00:58:42.758808 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:42.758874 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:42.761874 (+ 3000us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:42.762664 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:42.762679 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:42.763573 (+ 894us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:42.764547 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:42.764579 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:42.765951 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:42.765960 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:42.768421 (+ 2461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:42.768440 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:42.769323 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:42.769340 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:42.769708 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:42.770587 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:42.770624 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:42.773246 (+ 2622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:42.777585 (+ 4339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:42.777593 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:42.777608 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:42.778127 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:42.778636 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:42.778640 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:42.778643 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:42.778860 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:42.779668 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:42.779675 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:42.780068 (+ 393us) client_negotiation.cc:769] Sending connection context
0810 00:58:42.780331 (+ 263us) client_negotiation.cc:240] Negotiation successful
0810 00:58:42.780816 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":350,"threads_started":1}
I20260810 00:58:42.783815 30457 heartbeater.cc:347] Connected to a master server at 127.20.81.190:44873
I20260810 00:58:42.784157 30457 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:42.785029 30457 heartbeater.cc:511] Master 127.20.81.190:44873 requested a full tablet report, sending...
I20260810 00:58:42.787425 30246 ts_manager.cc:194] Registered new tserver with Master: 080086e2c5c8414d948a348684d713b0 (127.20.81.129:35173)
WARNING: no policy specified for kudu/127.20.81.130@KRBTEST.COM; defaulting to no policy
I20260810 00:58:42.788968 30246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:36331
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
I20260810 00:58:42.825856 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44873
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:43.182756 30463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:43.183102 30463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:43.183271 30463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:43.190773 30463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:43.190920 30463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:43.191092 30463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 00:58:43.200874 30463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:43.202280 30463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:43.203688 30463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:43.221709 30468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:43.221673 30472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:43.221645 30470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:43.222767 30463 server_base.cc:1034] running on GCE node
I20260810 00:58:43.223459 30463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:43.224493 30463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:43.225709 30463 hybrid_clock.cc:648] HybridClock initialized: now 1786323523225635 us; error 54 us; skew 500 ppm
I20260810 00:58:43.226073 30463 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:43 dist-test-slave-gp6n krb5kdc[30196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323523, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:43.229408 30463 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 00:58:43.231040 30463 webserver.cc:460] Webserver started at http://127.20.81.130:41573/ using document root <none> and password file <none>
I20260810 00:58:43.231745 30463 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:43.231877 30463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:43.232203 30463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:43.235260 30463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "a7a166a86d944861af4e9603e1b519cc"
format_stamp: "Formatted at 2026-08-10 00:58:43 on dist-test-slave-gp6n"
server_key: "9f0a1615cd60893a09dbfee075866969"
server_key_iv: "d01fafc84d868f117200458793afa87f"
server_key_version: "encryptionkey@0"
I20260810 00:58:43.236068 30463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "a7a166a86d944861af4e9603e1b519cc"
format_stamp: "Formatted at 2026-08-10 00:58:43 on dist-test-slave-gp6n"
server_key: "9f0a1615cd60893a09dbfee075866969"
server_key_iv: "d01fafc84d868f117200458793afa87f"
server_key_version: "encryptionkey@0"
I20260810 00:58:43.241241 30463 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 00:58:43.244742 30479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:43.246090 30463 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 00:58:43.246256 30463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "a7a166a86d944861af4e9603e1b519cc"
format_stamp: "Formatted at 2026-08-10 00:58:43 on dist-test-slave-gp6n"
server_key: "9f0a1615cd60893a09dbfee075866969"
server_key_iv: "d01fafc84d868f117200458793afa87f"
server_key_version: "encryptionkey@0"
I20260810 00:58:43.246505 30463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:43.293351 30463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:43.297950 30463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:43.298257 30463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:43.299266 30463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:43.300622 30463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:43.300727 30463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:43.300884 30463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:43.300959 30463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:43.334977 30463 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:34335
I20260810 00:58:43.335037 30600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:34335 every 8 connection(s)
I20260810 00:58:43.336284 30463 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 00:58:43.344918 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 30463
I20260810 00:58:43.345130 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 00:58:43.345587 20806 external_mini_cluster.cc:1428] Setting key b5203c3fe74aa31023f1d4ca5fac4343
Aug 10 00:58:43 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323523, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:58:43.357357 30603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:43.338923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:42917 (local address 127.20.81.190:44873)
0810 00:58:43.339349 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:43.339353 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:43.340351 (+ 998us) server_negotiation.cc:407] Connection header received
0810 00:58:43.342120 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:43.342124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:43.342199 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:43.342307 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:43.343816 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:43.344593 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:43.346340 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:43.346881 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:43.346985 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:43.351185 (+ 4200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:43.351190 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:43.351193 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:43.351245 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:43.353924 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:43.354945 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:43.354950 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:43.354952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:43.355084 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:43.355588 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:43.355591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:43.355593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:43.355864 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:43.355989 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:43.356880 (+ 891us) server_negotiation.cc:299] Negotiation successful
0810 00:58:43.357067 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":211,"threads_started":1}
I20260810 00:58:43.358914 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:43.339515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.130:42917)
0810 00:58:43.340189 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:43.340235 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:43.341881 (+ 1646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:43.342597 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:43.342605 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:43.343109 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:43.343661 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:43.343677 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:43.344743 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:43.344747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:43.346137 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:43.346150 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:43.347107 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:43.347114 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:43.347420 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:43.348099 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:43.348121 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:43.350915 (+ 2794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:43.354075 (+ 3160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:43.354088 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:43.354109 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:43.354821 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:43.355195 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:43.355202 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:43.355208 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:43.355473 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:43.355993 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:43.356005 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:43.356481 (+ 476us) client_negotiation.cc:769] Sending connection context
0810 00:58:43.356799 (+ 318us) client_negotiation.cc:240] Negotiation successful
0810 00:58:43.357406 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":272,"threads_started":1}
I20260810 00:58:43.360442 30601 heartbeater.cc:347] Connected to a master server at 127.20.81.190:44873
I20260810 00:58:43.360766 30601 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:43.361640 30601 heartbeater.cc:511] Master 127.20.81.190:44873 requested a full tablet report, sending...
I20260810 00:58:43.363283 30246 ts_manager.cc:194] Registered new tserver with Master: a7a166a86d944861af4e9603e1b519cc (127.20.81.130:34335)
I20260810 00:58:43.364162 30246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:42917
WARNING: no policy specified for kudu/127.20.81.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
I20260810 00:58:43.406685 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44873
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:43.743147 30607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:43.743454 30607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:43.743620 30607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:43.750707 30607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:43.750838 30607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:43.751052 30607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 00:58:43.763126 30607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:43.764345 30607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:43.765717 30607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:43.778832 30612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:43.778803 30613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:43.779037 30615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:43.779974 30607 server_base.cc:1034] running on GCE node
I20260810 00:58:43.780619 30607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:43.781579 30607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:43.782800 30607 hybrid_clock.cc:648] HybridClock initialized: now 1786323523782739 us; error 59 us; skew 500 ppm
I20260810 00:58:43.783121 30607 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:43 dist-test-slave-gp6n krb5kdc[30196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323523, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:43.786505 30607 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 00:58:43.788061 30607 webserver.cc:460] Webserver started at http://127.20.81.131:45857/ using document root <none> and password file <none>
I20260810 00:58:43.788765 30607 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:43.788905 30607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:43.789219 30607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:58:43.792510 30457 heartbeater.cc:503] Master 127.20.81.190:44873 was elected leader, sending a full tablet report...
I20260810 00:58:43.792383 30607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "f7c6ac609a0647b3a05c5fd5a5c09af6"
format_stamp: "Formatted at 2026-08-10 00:58:43 on dist-test-slave-gp6n"
server_key: "2ad02be865f0094780cd5c09924cd0fc"
server_key_iv: "2043166fcf9b5a83e1058401faa7d91c"
server_key_version: "encryptionkey@0"
I20260810 00:58:43.793448 30607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "f7c6ac609a0647b3a05c5fd5a5c09af6"
format_stamp: "Formatted at 2026-08-10 00:58:43 on dist-test-slave-gp6n"
server_key: "2ad02be865f0094780cd5c09924cd0fc"
server_key_iv: "2043166fcf9b5a83e1058401faa7d91c"
server_key_version: "encryptionkey@0"
I20260810 00:58:43.798872 30607 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 00:58:43.802307 30622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:43.803740 30607 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:58:43.803910 30607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "f7c6ac609a0647b3a05c5fd5a5c09af6"
format_stamp: "Formatted at 2026-08-10 00:58:43 on dist-test-slave-gp6n"
server_key: "2ad02be865f0094780cd5c09924cd0fc"
server_key_iv: "2043166fcf9b5a83e1058401faa7d91c"
server_key_version: "encryptionkey@0"
I20260810 00:58:43.804111 30607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:43.819351 30607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:43.823760 30607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:43.824061 30607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:43.825027 30607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:43.826508 30607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:43.826602 30607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:43.826745 30607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:43.826824 30607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:43.861419 30607 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:42859
I20260810 00:58:43.861469 30752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:42859 every 8 connection(s)
I20260810 00:58:43.863116 30607 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 00:58:43.872985 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 30607
I20260810 00:58:43.873204 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 00:58:43.873728 20806 external_mini_cluster.cc:1428] Setting key 00fa01c24fda236daae77623b866fad6
Aug 10 00:58:43 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323523, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:58:43.884616 30756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:43.865987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:37489 (local address 127.20.81.190:44873)
0810 00:58:43.866478 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:43.866485 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:43.867544 (+ 1059us) server_negotiation.cc:407] Connection header received
0810 00:58:43.869444 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:43.869448 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:43.869529 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:43.869629 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:43.871185 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:43.872021 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:43.874257 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:43.874878 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:43.875020 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:43.879085 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:43.879090 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:43.879093 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:43.879127 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:43.881718 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:43.882560 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:43.882564 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:43.882566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:43.882687 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:43.883104 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:43.883107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:43.883109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:43.883366 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:43.883488 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:43.884166 (+ 678us) server_negotiation.cc:299] Negotiation successful
0810 00:58:43.884333 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":211,"threads_started":1}
I20260810 00:58:43.885793 30757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:43.866519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.131:37489)
0810 00:58:43.867383 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:43.867435 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:43.869157 (+ 1722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:43.869837 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:43.869845 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:43.870366 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:43.871029 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:43.871046 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:43.872152 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:43.872155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:43.874101 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:43.874116 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:43.875253 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:43.875263 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:43.875607 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:43.876442 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:43.876465 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:43.878900 (+ 2435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:58:43.881855 (+ 2955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:43.881862 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:43.881874 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:43.882380 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:43.882787 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:58:43.882791 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:58:43.882794 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:58:43.882999 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:58:43.883483 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:58:43.883489 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:58:43.883808 (+ 319us) client_negotiation.cc:769] Sending connection context
0810 00:58:43.884048 (+ 240us) client_negotiation.cc:240] Negotiation successful
0810 00:58:43.884495 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":359,"threads_started":1}
I20260810 00:58:43.887207 30754 heartbeater.cc:347] Connected to a master server at 127.20.81.190:44873
I20260810 00:58:43.887586 30754 heartbeater.cc:464] Registering TS with master...
I20260810 00:58:43.888504 30754 heartbeater.cc:511] Master 127.20.81.190:44873 requested a full tablet report, sending...
I20260810 00:58:43.890132 30246 ts_manager.cc:194] Registered new tserver with Master: f7c6ac609a0647b3a05c5fd5a5c09af6 (127.20.81.131:42859)
I20260810 00:58:43.890976 30246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:37489
I20260810 00:58:43.901628 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.20.81.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.81.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.81.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.129.keytab.
Entry for principal oryx/127.20.81.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.129.keytab.
WARNING: no policy specified for oryx/127.20.81.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.81.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.81.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.130.keytab.
Entry for principal oryx/127.20.81.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.130.keytab.
WARNING: no policy specified for oryx/127.20.81.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.81.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.81.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.131.keytab.
Entry for principal oryx/127.20.81.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.131.keytab.
I20260810 00:58:44.091367 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 30314
Aug 10 00:58:44 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.129@KRBTEST.COM
I20260810 00:58:44.109180 30765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:44.092534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53640 (local address 127.20.81.129:35173)
0810 00:58:44.093292 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:44.093300 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:44.093384 (+ 84us) server_negotiation.cc:407] Connection header received
0810 00:58:44.093600 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:44.093607 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:44.093835 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:44.093987 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:44.095599 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:44.096666 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:44.098223 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:44.098910 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:44.099027 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:44.103281 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:44.103293 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:44.103308 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:44.103347 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:44.106277 (+ 2930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:44.106989 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:44.106997 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:44.106999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:44.107154 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:44.107631 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:44.107635 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:44.107637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:44.108060 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:44.108389 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:44.108622 (+ 233us) server_negotiation.cc:299] Negotiation successful
0810 00:58:44.108837 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":455,"threads_started":1}
I20260810 00:58:44.110514 30450 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:44.199108 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 30314
I20260810 00:58:44.221346 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 30463
Aug 10 00:58:44 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.130@KRBTEST.COM
I20260810 00:58:44.240811 30767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:44.222534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35490 (local address 127.20.81.130:34335)
0810 00:58:44.223222 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:44.223233 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:44.223272 (+ 39us) server_negotiation.cc:407] Connection header received
0810 00:58:44.223370 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:44.223377 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:44.223627 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:44.223822 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:44.225680 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:44.227582 (+ 1902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:44.229056 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:44.229683 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:44.229835 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:44.234700 (+ 4865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:44.234712 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:44.234727 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:44.234764 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:44.237823 (+ 3059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:44.238581 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:44.238591 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:44.238595 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:44.238723 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:44.239138 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:44.239142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:44.239146 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:44.239497 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:44.239804 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:44.240135 (+ 331us) server_negotiation.cc:299] Negotiation successful
0810 00:58:44.240427 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":321,"threads_started":1}
I20260810 00:58:44.242144 30595 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:44.327590 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 30463
I20260810 00:58:44.347159 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 30607
Aug 10 00:58:44 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.131@KRBTEST.COM
I20260810 00:58:44.364912 30769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:44.348243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46572 (local address 127.20.81.131:42859)
0810 00:58:44.348827 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:44.348835 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:44.348878 (+ 43us) server_negotiation.cc:407] Connection header received
0810 00:58:44.348938 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:44.348944 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:44.349149 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:44.349286 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:44.350689 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:44.351653 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:44.352958 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:44.353543 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:44.353649 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:44.358171 (+ 4522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:44.358183 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:44.358196 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:44.358230 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:44.362304 (+ 4074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:44.362982 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:44.362989 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:44.362992 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:44.363110 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:44.363468 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:44.363471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:44.363473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:44.363803 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:44.364018 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:44.364278 (+ 260us) server_negotiation.cc:299] Negotiation successful
0810 00:58:44.364493 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":296,"threads_started":1}
I20260810 00:58:44.366164 30747 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:44.448911 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 30607
I20260810 00:58:44.469247 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 30210
I20260810 00:58:44.470628 30280 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:58:44.575608 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 30210
I20260810 00:58:44.604166 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:44873
--webserver_interface=127.20.81.190
--webserver_port=45935
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:44873
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:58:44.942849 30772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:44.943176 30772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:44.943337 30772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:44.950657 30772 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:58:44.950807 30772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:44.950909 30772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:44.951005 30772 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:58:44.951092 30772 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:58:44.960960 30772 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:44873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:44873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=45935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:44.962330 30772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:44.963694 30772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:44.975587 30777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:44.975586 30781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:44.975672 30778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:44.976716 30772 server_base.cc:1034] running on GCE node
I20260810 00:58:44.977460 30772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:44.978977 30772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:44.980187 30772 hybrid_clock.cc:648] HybridClock initialized: now 1786323524980119 us; error 54 us; skew 500 ppm
I20260810 00:58:44.980532 30772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:44 dist-test-slave-gp6n krb5kdc[30196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323524, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:44.983728 30772 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 00:58:44.985214 30772 webserver.cc:460] Webserver started at http://127.20.81.190:45935/ using document root <none> and password file <none>
I20260810 00:58:44.985896 30772 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:44.986060 30772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:44.991391 30772 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 00:58:44.994227 30788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:44.995581 30772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 00:58:44.995761 30772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "587389e4d023465e94ded04f4a6ca8e1"
format_stamp: "Formatted at 2026-08-10 00:58:42 on dist-test-slave-gp6n"
server_key: "fd820f0bd46c07ebdc51772f13dfc158"
server_key_iv: "0e2b28c86252b16cd1677aa1c2c1611f"
server_key_version: "encryptionkey@0"
I20260810 00:58:44.996733 30772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:45.015466 30772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:45.019305 30772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:45.019580 30772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:45.038704 30772 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:44873
I20260810 00:58:45.038712 30846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:44873 every 8 connection(s)
I20260810 00:58:45.040277 30772 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:58:45.047168 30847 sys_catalog.cc:263] Verifying existing consensus state
I20260810 00:58:45.050354 30847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1: Bootstrap starting.
I20260810 00:58:45.050678 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 30772
I20260810 00:58:45.051507 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:35173
--local_ip_for_outbound_sockets=127.20.81.129
--tserver_master_addrs=127.20.81.190:44873
--webserver_port=37855
--webserver_interface=127.20.81.129
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.81.129
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.129.keytab with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:58:45.069665 30847 log.cc:824] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1: Log is configured to *not* fsync() on all Append() calls
I20260810 00:58:45.078579 30847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 00:58:45.079108 30847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1: Bootstrap complete.
I20260810 00:58:45.088372 30847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587389e4d023465e94ded04f4a6ca8e1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44873 } }
I20260810 00:58:45.089259 30847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 587389e4d023465e94ded04f4a6ca8e1, State: Initialized, Role: FOLLOWER
I20260810 00:58:45.089965 30847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587389e4d023465e94ded04f4a6ca8e1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44873 } }
I20260810 00:58:45.090224 30847 raft_consensus.cc:388] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:58:45.090344 30847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:58:45.090587 30847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [term 1 FOLLOWER]: Advancing to term 2
I20260810 00:58:45.094386 30847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587389e4d023465e94ded04f4a6ca8e1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44873 } }
I20260810 00:58:45.094942 30847 leader_election.cc:302] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 587389e4d023465e94ded04f4a6ca8e1; no voters:
I20260810 00:58:45.095589 30847 leader_election.cc:288] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 00:58:45.095726 30852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 00:58:45.097047 30852 raft_consensus.cc:686] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [term 2 LEADER]: Becoming Leader. State: Replica: 587389e4d023465e94ded04f4a6ca8e1, State: Running, Role: LEADER
I20260810 00:58:45.097676 30852 consensus_queue.cc:237] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587389e4d023465e94ded04f4a6ca8e1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44873 } }
I20260810 00:58:45.097899 30847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:58:45.100294 30854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 587389e4d023465e94ded04f4a6ca8e1. Latest consensus state: current_term: 2 leader_uuid: "587389e4d023465e94ded04f4a6ca8e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587389e4d023465e94ded04f4a6ca8e1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44873 } } }
I20260810 00:58:45.100684 30854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:45.101310 30853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "587389e4d023465e94ded04f4a6ca8e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587389e4d023465e94ded04f4a6ca8e1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44873 } } }
I20260810 00:58:45.101624 30853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1 [sys.catalog]: This master's current role is: LEADER
I20260810 00:58:45.107158 30857 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:58:45.114020 30857 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:58:45.115895 30857 catalog_manager.cc:1259] Loaded cluster ID: f11869958bc44e58bcb397809ac63495
I20260810 00:58:45.116091 30857 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:58:45.121273 30857 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:58:45.123286 30857 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 587389e4d023465e94ded04f4a6ca8e1: Loaded TSK: 0
I20260810 00:58:45.125260 30857 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 00:58:45.426206 30850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:45.426599 30850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:45.426769 30850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:45.434727 30850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:45.434875 30850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:45.435053 30850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 00:58:45.445344 30850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.129.keytab
--principal=oryx/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:35173
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=37855
--tserver_master_addrs=127.20.81.190:44873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:45.446805 30850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:45.448187 30850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:45.464448 30877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:45.465083 30876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:45.466730 30880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:45.470603 30850 server_base.cc:1034] running on GCE node
I20260810 00:58:45.471375 30850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:45.472446 30850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:45.473708 30850 hybrid_clock.cc:648] HybridClock initialized: now 1786323525473644 us; error 55 us; skew 500 ppm
I20260810 00:58:45.474102 30850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:45 dist-test-slave-gp6n krb5kdc[30196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323525, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:45.477794 30850 init.cc:377] Logged in from keytab as oryx/127.20.81.129@KRBTEST.COM (short username oryx)
I20260810 00:58:45.479743 30850 webserver.cc:460] Webserver started at http://127.20.81.129:37855/ using document root <none> and password file <none>
I20260810 00:58:45.480532 30850 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:45.480687 30850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:45.486943 30850 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 00:58:45.489988 30889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:45.491549 30850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:58:45.491724 30850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "080086e2c5c8414d948a348684d713b0"
format_stamp: "Formatted at 2026-08-10 00:58:42 on dist-test-slave-gp6n"
server_key: "7083e52eac8f5e77f23be70e88d52bc8"
server_key_iv: "970ca24faafcc0c69ce1888e59b048e6"
server_key_version: "encryptionkey@0"
I20260810 00:58:45.492761 30850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:45.517845 30850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:45.522675 30850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:45.523003 30850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:45.524021 30850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:45.525436 30850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:45.525530 30850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:45.525619 30850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:45.525683 30850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:45.562600 30850 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:35173
I20260810 00:58:45.562687 31010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:35173 every 8 connection(s)
I20260810 00:58:45.564180 30850 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 00:58:45.571038 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 30850
I20260810 00:58:45.571653 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:34335
--local_ip_for_outbound_sockets=127.20.81.130
--tserver_master_addrs=127.20.81.190:44873
--webserver_port=41573
--webserver_interface=127.20.81.130
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.81.130
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.130.keytab with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 00:58:45 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:45 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 00:58:46.041824 31015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:46.042335 31015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:46.042610 31015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:46.058812 31015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:46.059001 31015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:46.059283 31015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 00:58:46.081029 31015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.130.keytab
--principal=oryx/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:34335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=41573
--tserver_master_addrs=127.20.81.190:44873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:46.083132 31015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:46.085116 31015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:46.112764 31022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:46.114176 31023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:46.122149 31026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:46.122911 31015 server_base.cc:1034] running on GCE node
I20260810 00:58:46.123839 31015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:46.125224 31015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:46.126796 31015 hybrid_clock.cc:648] HybridClock initialized: now 1786323526126703 us; error 90 us; skew 500 ppm
I20260810 00:58:46.127254 31015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:46 dist-test-slave-gp6n krb5kdc[30196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323526, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:46.133034 31015 init.cc:377] Logged in from keytab as oryx/127.20.81.130@KRBTEST.COM (short username oryx)
I20260810 00:58:46.135964 31015 webserver.cc:460] Webserver started at http://127.20.81.130:41573/ using document root <none> and password file <none>
I20260810 00:58:46.137552 31015 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:46.137797 31015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:46.149296 31015 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.009s sys 0.000s
I20260810 00:58:46.154685 31033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:46.157256 31015 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260810 00:58:46.157555 31015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "a7a166a86d944861af4e9603e1b519cc"
format_stamp: "Formatted at 2026-08-10 00:58:43 on dist-test-slave-gp6n"
server_key: "9f0a1615cd60893a09dbfee075866969"
server_key_iv: "d01fafc84d868f117200458793afa87f"
server_key_version: "encryptionkey@0"
I20260810 00:58:46.159859 31015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:46.212038 31015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:46.220165 31015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:46.220614 31015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:46.222491 31015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:46.225052 31015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:46.225281 31015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 00:58:46.225461 31015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:46.225574 31015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:46.264891 31163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:34335 every 8 connection(s)
I20260810 00:58:46.264899 31015 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:34335
I20260810 00:58:46.267654 31015 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 00:58:46.270586 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 31015
I20260810 00:58:46.271191 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:42859
--local_ip_for_outbound_sockets=127.20.81.131
--tserver_master_addrs=127.20.81.190:44873
--webserver_port=45857
--webserver_interface=127.20.81.131
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.81.131
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.131.keytab with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 00:58:46 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:46 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 00:58:46.630960 31168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:58:46.631280 31168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:58:46.631467 31168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:58:46.639199 31168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:58:46.639353 31168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:58:46.639535 31168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 00:58:46.649267 31168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/krb5kdc/oryx_127.20.81.131.keytab
--principal=oryx/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:42859
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=45857
--tserver_master_addrs=127.20.81.190:44873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:58:46.650743 31168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:58:46.652070 31168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:58:46.675223 31175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:46.676785 31174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:58:46.677270 31177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:58:46.677770 31168 server_base.cc:1034] running on GCE node
I20260810 00:58:46.678529 31168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:58:46.679563 31168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:58:46.680779 31168 hybrid_clock.cc:648] HybridClock initialized: now 1786323526680714 us; error 60 us; skew 500 ppm
I20260810 00:58:46.681128 31168 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:58:46 dist-test-slave-gp6n krb5kdc[30196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323526, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:58:46.684501 31168 init.cc:377] Logged in from keytab as oryx/127.20.81.131@KRBTEST.COM (short username oryx)
I20260810 00:58:46.687623 31168 webserver.cc:460] Webserver started at http://127.20.81.131:45857/ using document root <none> and password file <none>
I20260810 00:58:46.688357 31168 fs_manager.cc:359] Metadata directory not provided
I20260810 00:58:46.688539 31168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:58:46.694165 31168 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 00:58:46.696998 31185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:46.698788 31168 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:58:46.698948 31168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "f7c6ac609a0647b3a05c5fd5a5c09af6"
format_stamp: "Formatted at 2026-08-10 00:58:43 on dist-test-slave-gp6n"
server_key: "2ad02be865f0094780cd5c09924cd0fc"
server_key_iv: "2043166fcf9b5a83e1058401faa7d91c"
server_key_version: "encryptionkey@0"
I20260810 00:58:46.700111 31168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:58:46.740566 31168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:58:46.744987 31168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:58:46.745285 31168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:58:46.746387 31168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:58:46.747882 31168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:58:46.748003 31168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:46.748134 31168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:58:46.748220 31168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:58:46.801676 31168 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:42859
I20260810 00:58:46.801726 31310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:42859 every 8 connection(s)
I20260810 00:58:46.803381 31168 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 00:58:46.810940 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 31168
Aug 10 00:58:46 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:46 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:46.837891 31315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:46.812429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57754 (local address 127.20.81.190:44873)
0810 00:58:46.816005 (+ 3576us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:46.816013 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:46.816034 (+ 21us) server_negotiation.cc:407] Connection header received
0810 00:58:46.816088 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:46.816095 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:46.816156 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:46.816251 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:46.819132 (+ 2881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:46.820506 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:46.822461 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:46.823116 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:46.823219 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:46.827386 (+ 4167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:58:46.827402 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:58:46.827427 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:58:46.827476 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:58:46.831913 (+ 4437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:46.832904 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:46.832916 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:46.832928 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:46.833154 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:58:46.833645 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:58:46.833653 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:58:46.833660 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:58:46.834354 (+ 694us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:58:46.834709 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0810 00:58:46.836255 (+ 1546us) server_negotiation.cc:299] Negotiation successful
0810 00:58:46.836745 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":133,"server-negotiator.queue_time_us":3430,"thread_start_us":1751,"threads_started":1}
W20260810 00:58:47.140477 31014 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.561s user 0.005s sys 0.013s
W20260810 00:58:47.142828 31013 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:45.567408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:36665 (local address 127.20.81.190:44873)
0810 00:58:45.568369 (+ 961us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:45.568387 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:45.568611 (+ 224us) server_negotiation.cc:407] Connection header received
0810 00:58:45.571134 (+ 2523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:45.571155 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:45.571655 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:45.572140 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:45.573821 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:45.575306 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:45.577074 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:45.577772 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:45.577997 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:47.142256 (+1564259us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.129:36665: BlockingRecv error: recv got EOF from 127.20.81.129:36665 (error 108)
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":365,"threads_started":1}
I20260810 00:58:47.144045 31014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:45.567504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.129:36665)
0810 00:58:45.568406 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:45.568486 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:45.570505 (+ 2019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:45.572305 (+ 1800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:45.572320 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:45.573016 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:45.573605 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:45.573625 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:45.575500 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:45.575504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:45.576887 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:45.576901 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:45.578055 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:45.578064 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:45.578368 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:45.579240 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:45.579269 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:47.141819 (+1562550us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":371,"threads_started":1}
W20260810 00:58:47.145921 31011 heartbeater.cc:644] Failed to heartbeat to 127.20.81.190:44873 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.81.190:44873: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Aug 10 00:58:47 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:47 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 00:58:47.172037 31013 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:47.147565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:41043 (local address 127.20.81.190:44873)
0810 00:58:47.148083 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:47.148093 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:47.148706 (+ 613us) server_negotiation.cc:407] Connection header received
0810 00:58:47.149051 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:47.149062 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:47.149194 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:47.149386 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:47.150685 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.152539 (+ 1854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.155285 (+ 2746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.156388 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.156587 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:47.171626 (+ 15039us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.129:41043: BlockingRecv error: recv got EOF from 127.20.81.129:41043 (error 108)
Metrics: {"server-negotiator.queue_time_us":324}
I20260810 00:58:47.175570 31014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:47.147067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.129:41043)
0810 00:58:47.148580 (+ 1513us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:47.148611 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:47.148855 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:47.149411 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:47.149420 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:47.150125 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:47.150479 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:47.150498 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.152745 (+ 2247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.152754 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:47.155088 (+ 2334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:47.155110 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.156643 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.156652 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:47.156933 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:47.164613 (+ 7680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:47.164656 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:47.174957 (+ 10301us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1054,"mutex_wait_us":579}
Aug 10 00:58:47 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:47 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:47.192786 31014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:47.176627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.129:58673)
0810 00:58:47.176962 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:47.176986 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:47.177123 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:47.177703 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:47.177711 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:47.178332 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:47.178661 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:47.178679 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.180890 (+ 2211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.180899 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:47.183245 (+ 2346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:47.183266 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.184772 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.184782 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:47.185072 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:47.186041 (+ 969us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:47.186077 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:47.192295 (+ 6218us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176}
W20260810 00:58:47.193042 31013 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:47.176781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:58673 (local address 127.20.81.190:44873)
0810 00:58:47.177026 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:47.177037 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:47.177066 (+ 29us) server_negotiation.cc:407] Connection header received
0810 00:58:47.177322 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:47.177331 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:47.177481 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:47.177634 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:47.178860 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.180683 (+ 1823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.183465 (+ 2782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.184547 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.184746 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:47.192626 (+ 7880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.129:58673: BlockingRecv error: recv got EOF from 127.20.81.129:58673 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260810 00:58:47.193800 31011 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 00:58:47.731550 31167 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.431s user 0.002s sys 0.017s
W20260810 00:58:47.732875 31166 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:46.271780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:36243 (local address 127.20.81.190:44873)
0810 00:58:46.285568 (+ 13788us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:46.285576 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:46.286239 (+ 663us) server_negotiation.cc:407] Connection header received
0810 00:58:46.289544 (+ 3305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:46.289555 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:46.289702 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:46.289885 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:46.292258 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:46.294047 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:46.296957 (+ 2910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:46.297950 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:46.298128 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:47.732601 (+1434473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.130:36243: BlockingRecv error: recv got EOF from 127.20.81.130:36243 (error 108)
Metrics: {"server-negotiator.queue_time_us":13613,"thread_start_us":13473,"threads_started":1}
I20260810 00:58:47.733687 31167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:46.272105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.130:36243)
0810 00:58:46.286029 (+ 13924us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:46.286105 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:46.289140 (+ 3035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:46.290181 (+ 1041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:46.290197 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:46.291094 (+ 897us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:46.292022 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:46.292052 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:46.294255 (+ 2203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:46.294263 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:46.296766 (+ 2503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:46.296784 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:46.298228 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:46.298238 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:46.298615 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:46.300598 (+ 1983us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:46.300645 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:47.732418 (+1431773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":13535,"thread_start_us":13219,"threads_started":1}
W20260810 00:58:47.735124 31164 heartbeater.cc:644] Failed to heartbeat to 127.20.81.190:44873 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.81.190:44873: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Aug 10 00:58:47 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:47 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:47.746640 31167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:47.736039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.130:44763)
0810 00:58:47.736305 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:47.736341 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:47.736469 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:47.736806 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:47.736811 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:47.737237 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:47.737422 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:47.737432 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.738625 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.738632 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:47.739684 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:47.739696 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.740542 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.740548 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:47.740742 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:47.741404 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:47.741424 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:47.746284 (+ 4860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157}
W20260810 00:58:47.746822 31166 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:47.736272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:44763 (local address 127.20.81.190:44873)
0810 00:58:47.736505 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:47.736514 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:47.736537 (+ 23us) server_negotiation.cc:407] Connection header received
0810 00:58:47.736595 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:47.736601 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:47.736672 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:47.736781 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:47.737571 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.738479 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.739834 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.740336 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.740482 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:47.746544 (+ 6062us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.130:44763: BlockingRecv error: recv got EOF from 127.20.81.130:44763 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 10 00:58:47 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:47 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:47.759013 31167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:47.747596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.130:51695)
0810 00:58:47.747757 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:47.747776 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:47.747868 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:47.748918 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:47.748923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:47.749255 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:47.749438 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:47.749447 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.750515 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.750521 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:47.751585 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:47.751597 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.752390 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.752396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:47.752590 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:47.753220 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:47.753238 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:47.758696 (+ 5458us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 00:58:47.759246 31166 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:47.748237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:51695 (local address 127.20.81.190:44873)
0810 00:58:47.748422 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:47.748427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:47.748441 (+ 14us) server_negotiation.cc:407] Connection header received
0810 00:58:47.748502 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:47.748506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:47.748591 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:47.748706 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:47.749582 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.750331 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.751742 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.752215 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.752347 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:47.758929 (+ 6582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.130:51695: BlockingRecv error: recv got EOF from 127.20.81.130:51695 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 00:58:47.759467 31164 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 00:58:47.940163 31314 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.115s user 0.004s sys 0.014s
W20260810 00:58:47.941398 31317 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:46.808639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:42677 (local address 127.20.81.190:44873)
0810 00:58:46.814485 (+ 5846us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:46.814491 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:46.814511 (+ 20us) server_negotiation.cc:407] Connection header received
0810 00:58:46.814574 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:46.814579 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:46.814655 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:46.814789 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:46.818922 (+ 4133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:46.820506 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:46.822974 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:46.823589 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:46.824655 (+ 1066us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:47.941150 (+1116495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.131:42677: BlockingRecv error: recv got EOF from 127.20.81.131:42677 (error 108)
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":5671,"thread_start_us":7080,"threads_started":1}
I20260810 00:58:47.941955 31314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:46.807824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.131:42677)
0810 00:58:46.810301 (+ 2477us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:46.810358 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:46.812920 (+ 2562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:46.816770 (+ 3850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:46.816785 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:46.817622 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:46.818662 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:46.818693 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:46.820691 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:46.820696 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:46.822758 (+ 2062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:46.822774 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:46.823858 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:46.823869 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:46.824235 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:46.825003 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:46.825036 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:47.940959 (+1115923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":2144,"thread_start_us":1867,"threads_started":1}
W20260810 00:58:47.942938 31311 heartbeater.cc:644] Failed to heartbeat to 127.20.81.190:44873 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.81.190:44873: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Aug 10 00:58:47 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:47 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:47.953114 31314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:47.943659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.131:56091)
0810 00:58:47.943885 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:47.943906 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:47.944051 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:47.944441 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:47.944445 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:47.944961 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:47.945145 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:47.945156 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.946448 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.946454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:47.947501 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:47.947514 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.948337 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.948344 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:47.948511 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:47.949189 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:47.949209 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:47.952812 (+ 3603us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":67}
W20260810 00:58:47.953272 31317 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:47.943737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:56091 (local address 127.20.81.190:44873)
0810 00:58:47.943899 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:47.943906 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:47.943983 (+ 77us) server_negotiation.cc:407] Connection header received
0810 00:58:47.944176 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:47.944181 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:47.944253 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:47.944416 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:47.945399 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.946267 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.947674 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.948118 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.948259 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:47.953038 (+ 4779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.131:56091: BlockingRecv error: recv got EOF from 127.20.81.131:56091 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 10 00:58:47 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:47 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:47.961923 31314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:47.953839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.131:42181)
0810 00:58:47.953972 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:47.953989 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:47.954095 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:47.954522 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:47.954529 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:47.954813 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:47.954953 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:47.954962 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.955934 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.955939 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:47.956892 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:47.956903 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.957604 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.957610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:47.957786 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:47.958347 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:47.958364 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:47.961708 (+ 3344us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 00:58:47.962169 31317 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:47.953919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:42181 (local address 127.20.81.190:44873)
0810 00:58:47.954216 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:47.954221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:47.954236 (+ 15us) server_negotiation.cc:407] Connection header received
0810 00:58:47.954280 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:47.954283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:47.954341 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:47.954464 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:47.955085 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.955801 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.957033 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:47.957463 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:47.957592 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:47.961882 (+ 4290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.131:42181: BlockingRecv error: recv got EOF from 127.20.81.131:42181 (error 108)
Metrics: {"server-negotiator.queue_time_us":172}
W20260810 00:58:47.962249 31311 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 00:58:48 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:48 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:48.204861 31319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:48.194544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.129:42301)
0810 00:58:48.195241 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:48.195274 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:48.195422 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:48.195815 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:48.195820 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:48.196461 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:48.196624 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:48.196634 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:48.197935 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:48.197941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:48.199197 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:48.199211 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:48.200055 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:48.200060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:48.200192 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:48.200898 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:48.200916 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:48.204536 (+ 3620us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":483,"threads_started":1}
W20260810 00:58:48.205266 31317 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:48.194635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:42301 (local address 127.20.81.190:44873)
0810 00:58:48.194905 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:48.194916 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:48.195378 (+ 462us) server_negotiation.cc:407] Connection header received
0810 00:58:48.195542 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:48.195547 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:48.195656 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:48.195778 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:48.196785 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:48.197739 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:48.199350 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:48.199868 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:48.199969 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:48.204785 (+ 4816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.129:42301: BlockingRecv error: recv got EOF from 127.20.81.129:42301 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 10 00:58:48 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:48 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:48.770020 31320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:48.760080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.130:47969)
0810 00:58:48.760670 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:48.760689 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:48.760809 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:48.761200 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:48.761204 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:48.761707 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:48.761879 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:48.761890 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:48.762905 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:48.762909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:48.764039 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:48.764053 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:48.764771 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:48.764776 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:48.764908 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:48.765713 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:48.765733 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:48.769642 (+ 3909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":302,"threads_started":1}
W20260810 00:58:48.770252 31321 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:48.760161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:47969 (local address 127.20.81.190:44873)
0810 00:58:48.760930 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:48.760935 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:48.760950 (+ 15us) server_negotiation.cc:407] Connection header received
0810 00:58:48.760996 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:48.761000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:48.761057 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:48.761144 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:48.762018 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:48.762767 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:48.764196 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:48.764598 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:48.764688 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:48.769907 (+ 5219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.130:47969: BlockingRecv error: recv got EOF from 127.20.81.130:47969 (error 108)
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":561,"threads_started":1}
Aug 10 00:58:48 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:48 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:48.972752 31322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:48.962924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.131:37697)
0810 00:58:48.963471 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:48.963489 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:48.963606 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:48.963967 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:48.963971 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:48.964433 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:48.964595 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:48.964604 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:48.965678 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:48.965682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:48.966738 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:48.966750 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:48.967518 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:48.967522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:48.967646 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:48.968366 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:48.968385 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:48.972428 (+ 4043us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":284,"threads_started":1}
W20260810 00:58:48.973013 31321 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:48.963000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:37697 (local address 127.20.81.190:44873)
0810 00:58:48.963242 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:48.963248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:48.963599 (+ 351us) server_negotiation.cc:407] Connection header received
0810 00:58:48.963710 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:48.963714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:48.963786 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:48.963892 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:48.964762 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:48.965533 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:48.966889 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:48.967346 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:48.967458 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:48.972778 (+ 5320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.131:37697: BlockingRecv error: recv got EOF from 127.20.81.131:37697 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 10 00:58:49 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:49 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:49.215492 31323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:49.205797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.129:36877)
0810 00:58:49.206341 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:49.206363 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:49.206554 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:49.206929 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:49.206934 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:49.207408 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:49.207585 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:49.207594 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:49.208647 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:49.208653 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:49.209605 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:49.209617 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:49.210362 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:49.210368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:49.210571 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:49.211348 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:49.211372 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:49.215212 (+ 3840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":301,"threads_started":1}
W20260810 00:58:49.215737 31321 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:49.205877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:36877 (local address 127.20.81.190:44873)
0810 00:58:49.206125 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:49.206133 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:49.206468 (+ 335us) server_negotiation.cc:407] Connection header received
0810 00:58:49.206683 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:49.206689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:49.206768 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:49.206871 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:49.207740 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:49.208503 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:49.209751 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:49.210188 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:49.210343 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:49.215444 (+ 5101us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.129:36877: BlockingRecv error: recv got EOF from 127.20.81.129:36877 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 10 00:58:49 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:49 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:49.780375 31324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:49.771168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.130:35361)
0810 00:58:49.771801 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:49.771826 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:49.771964 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:49.772315 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:49.772319 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:49.772878 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:49.773060 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:49.773070 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:49.774077 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:49.774083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:49.775038 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:49.775049 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:49.775769 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:49.775775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:49.775961 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:49.776553 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:49.776571 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:49.780028 (+ 3457us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":317,"threads_started":1}
W20260810 00:58:49.780508 31325 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:49.771267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:35361 (local address 127.20.81.190:44873)
0810 00:58:49.772030 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:49.772034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:49.772050 (+ 16us) server_negotiation.cc:407] Connection header received
0810 00:58:49.772092 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:49.772095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:49.772161 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:49.772282 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:49.773212 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:49.773948 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:49.775177 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:49.775620 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:49.775752 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:49.780233 (+ 4481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.130:35361: BlockingRecv error: recv got EOF from 127.20.81.130:35361 (error 108)
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":627,"threads_started":1}
Aug 10 00:58:49 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:49 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:49.983155 31326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:49.973757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.131:60791)
0810 00:58:49.974555 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:49.974580 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:49.974722 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:49.975046 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:49.975049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:49.975478 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:49.975669 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:49.975678 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:49.976649 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:49.976653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:49.977613 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:49.977624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:49.978352 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:49.978356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:49.978517 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:49.979175 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:49.979194 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:49.982890 (+ 3696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":382,"threads_started":1}
W20260810 00:58:49.983390 31325 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:49.973941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:60791 (local address 127.20.81.190:44873)
0810 00:58:49.974174 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:49.974180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:49.974701 (+ 521us) server_negotiation.cc:407] Connection header received
0810 00:58:49.974817 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:49.974820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:49.974882 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:49.974969 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:49.975798 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:49.976529 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:49.977749 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:49.978187 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:49.978283 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:49.983130 (+ 4847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.131:60791: BlockingRecv error: recv got EOF from 127.20.81.131:60791 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 10 00:58:50 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:50 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:50.227912 31327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:50.216547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.129:52249)
0810 00:58:50.217175 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:50.217199 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:50.217353 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:50.217843 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:50.217850 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:50.218429 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:50.218719 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:50.218735 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:50.219923 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:50.219929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:50.221054 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:50.221071 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:50.222100 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:50.222111 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:50.222265 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:50.223065 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:50.223088 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:50.227614 (+ 4526us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":296,"threads_started":1}
W20260810 00:58:50.228084 31325 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:50.216627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:52249 (local address 127.20.81.190:44873)
0810 00:58:50.216878 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:50.216885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:50.217308 (+ 423us) server_negotiation.cc:407] Connection header received
0810 00:58:50.217533 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:50.217538 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:50.217642 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:50.217775 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:50.218925 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:50.219756 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:50.221225 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:50.221873 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:50.222055 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:50.227811 (+ 5756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.129:52249: BlockingRecv error: recv got EOF from 127.20.81.129:52249 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 10 00:58:50 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:50 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:50.791152 31328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:50.781388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.130:35101)
0810 00:58:50.781905 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:50.781923 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:50.782031 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:50.782605 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:50.782608 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:50.782954 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:50.783133 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:50.783142 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:50.784158 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:50.784162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:50.785099 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:50.785111 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:50.786178 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:50.786185 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:50.786379 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:50.787170 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:50.787189 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:50.790906 (+ 3717us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":251,"threads_started":1}
W20260810 00:58:50.791494 31329 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:50.781469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:35101 (local address 127.20.81.190:44873)
0810 00:58:50.782252 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:50.782258 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:50.782273 (+ 15us) server_negotiation.cc:407] Connection header received
0810 00:58:50.782327 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:50.782330 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:50.782467 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:50.782568 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:50.783263 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:50.784021 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:50.785278 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:50.785960 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:50.786113 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:50.791147 (+ 5034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.130:35101: BlockingRecv error: recv got EOF from 127.20.81.130:35101 (error 108)
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":575,"threads_started":1}
Aug 10 00:58:50 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:50 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:50.993403 31330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:50.984142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.131:57691)
0810 00:58:50.984683 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:50.984699 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:50.984851 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:50.985206 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:50.985211 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:50.985597 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:50.985771 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:50.985782 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:50.986862 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:50.986869 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:50.987791 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:50.987803 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:50.988514 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:50.988520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:50.988693 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:50.989383 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:50.989402 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:50.993134 (+ 3732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":284,"threads_started":1}
W20260810 00:58:50.993692 31329 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:50.984234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:57691 (local address 127.20.81.190:44873)
0810 00:58:50.984468 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:50.984476 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:50.984786 (+ 310us) server_negotiation.cc:407] Connection header received
0810 00:58:50.984991 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:50.984997 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:50.985069 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:50.985168 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:50.985932 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:50.986732 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:50.987941 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:50.988370 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:50.988499 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:50.993336 (+ 4837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.131:57691: BlockingRecv error: recv got EOF from 127.20.81.131:57691 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 10 00:58:51 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:51 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:51.238160 31331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:51.228971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.129:51923)
0810 00:58:51.229518 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:51.229535 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:51.229690 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:51.230056 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:51.230062 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:51.230513 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:51.230706 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:51.230715 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:51.231763 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:51.231771 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:51.232798 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:51.232811 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:51.233515 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:51.233520 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:51.233709 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:51.234299 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:51.234317 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:51.237876 (+ 3559us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":329,"threads_started":1}
W20260810 00:58:51.238547 31329 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:51.229038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:51923 (local address 127.20.81.190:44873)
0810 00:58:51.229330 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:51.229338 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:51.229626 (+ 288us) server_negotiation.cc:407] Connection header received
0810 00:58:51.229829 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:51.229833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:51.229894 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:51.230011 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:51.230852 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:51.231624 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:51.232948 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:51.233367 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:51.233502 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:51.238162 (+ 4660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.129:51923: BlockingRecv error: recv got EOF from 127.20.81.129:51923 (error 108)
Metrics: {"server-negotiator.queue_time_us":129}
Aug 10 00:58:51 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:51 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:51.802742 31333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:51.792229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.130:49347)
0810 00:58:51.793039 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:51.793055 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:51.793166 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:51.793521 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:51.793525 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:51.793911 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:51.794070 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:51.794078 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:51.795291 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:51.795295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:51.796371 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:51.796384 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:51.797193 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:51.797197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:51.797368 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:51.798018 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:51.798035 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:51.802342 (+ 4307us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":598,"threads_started":1}
W20260810 00:58:51.802879 31332 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:51.792305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:49347 (local address 127.20.81.190:44873)
0810 00:58:51.792746 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:51.792751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:51.793139 (+ 388us) server_negotiation.cc:407] Connection header received
0810 00:58:51.793268 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:51.793271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:51.793363 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:51.793471 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:51.794220 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:51.795153 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:51.796511 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:51.797062 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:51.797167 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:51.802587 (+ 5420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.130:49347: BlockingRecv error: recv got EOF from 127.20.81.130:49347 (error 108)
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":250,"threads_started":1}
Aug 10 00:58:51 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:52 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:52.003971 31334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:51.994537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.131:53147)
0810 00:58:51.995096 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:51.995112 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:51.995225 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:51.995527 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:51.995531 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:51.995945 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:51.996112 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:51.996121 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:51.997366 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:51.997370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:51.998599 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:51.998614 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:51.999429 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:51.999434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:51.999561 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:52.000226 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:52.000245 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:52.003695 (+ 3450us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":316,"threads_started":1}
W20260810 00:58:52.004248 31332 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:51.994579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:53147 (local address 127.20.81.190:44873)
0810 00:58:51.994773 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:51.994778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:51.995191 (+ 413us) server_negotiation.cc:407] Connection header received
0810 00:58:51.995321 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:51.995325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:51.995388 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:51.995474 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:51.996285 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:51.997204 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:51.998793 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:51.999263 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:51.999374 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:52.003989 (+ 4615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.131:53147: BlockingRecv error: recv got EOF from 127.20.81.131:53147 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 10 00:58:52 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:52 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:52.254033 31335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:52.239140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.129:46519)
0810 00:58:52.239704 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:52.239724 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:52.239856 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:52.240229 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:52.240235 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:52.240673 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:52.240835 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:52.240844 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:52.241901 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:52.241907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:52.242943 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:52.242955 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:52.243982 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:52.243988 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:52.244185 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:52.249634 (+ 5449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:52.249655 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:52.253792 (+ 4137us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":262,"threads_started":1}
W20260810 00:58:52.254292 31332 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:52.239211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:46519 (local address 127.20.81.190:44873)
0810 00:58:52.239448 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:52.239457 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:52.239787 (+ 330us) server_negotiation.cc:407] Connection header received
0810 00:58:52.239992 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:52.239998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:52.240089 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:52.240194 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:52.240989 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:52.241757 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:52.243113 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:52.243703 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:52.243968 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:52.254004 (+ 10036us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.129:46519: BlockingRecv error: recv got EOF from 127.20.81.129:46519 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 10 00:58:52 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:52 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:52.813477 31336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:52.803979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.130:44491)
0810 00:58:52.804607 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:52.804631 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:52.804797 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:52.805165 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:52.805169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:52.805619 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:52.805790 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:52.805800 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:52.806816 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:52.806820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:52.807893 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:52.807905 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:52.808661 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:52.808666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:52.808804 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:52.809465 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:52.809485 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:52.813242 (+ 3757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":259,"threads_started":1}
W20260810 00:58:52.813901 31338 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:52.804160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:44491 (local address 127.20.81.190:44873)
0810 00:58:52.804825 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:52.804834 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:52.804858 (+ 24us) server_negotiation.cc:407] Connection header received
0810 00:58:52.804928 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:52.804932 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:52.805001 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:52.805092 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:52.805932 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:52.806675 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:52.808066 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:52.808505 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:52.808618 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:52.813544 (+ 4926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.130:44491: BlockingRecv error: recv got EOF from 127.20.81.130:44491 (error 108)
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":401,"threads_started":1}
Aug 10 00:58:53 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:53 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:53.014515 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:53.004971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.131:57491)
0810 00:58:53.005496 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:53.005513 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:53.005682 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:53.006049 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:53.006054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:53.006603 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:53.006753 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:53.006763 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:53.007885 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:53.007890 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:53.009051 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:53.009069 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:53.010119 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:53.010123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:53.010243 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:53.010872 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:53.010889 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:53.014190 (+ 3301us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":280,"threads_started":1}
W20260810 00:58:53.014643 31338 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:53.005060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:57491 (local address 127.20.81.190:44873)
0810 00:58:53.005293 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:53.005301 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:53.005601 (+ 300us) server_negotiation.cc:407] Connection header received
0810 00:58:53.005827 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:53.005832 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:53.005906 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:53.006013 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:53.006902 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:53.007735 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:53.009252 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:53.009965 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:53.010072 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:53.014428 (+ 4356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.131:57491: BlockingRecv error: recv got EOF from 127.20.81.131:57491 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 10 00:58:53 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:53 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:53.264231 31340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:53.254979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.129:50217)
0810 00:58:53.255514 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:53.255532 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:53.255657 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:53.255954 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:53.255958 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:53.256439 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:53.256614 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:53.256622 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:53.257643 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:53.257647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:53.258705 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:53.258718 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:53.259424 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:53.259429 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:53.259561 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:53.260220 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:53.260239 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:53.263997 (+ 3758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":275,"threads_started":1}
W20260810 00:58:53.264447 31338 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:53.255137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:50217 (local address 127.20.81.190:44873)
0810 00:58:53.255387 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:53.255393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:53.255618 (+ 225us) server_negotiation.cc:407] Connection header received
0810 00:58:53.255744 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:53.255748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:53.255823 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:53.255905 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:53.256744 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:53.257495 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:53.258840 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:53.259259 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:53.259354 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:53.264190 (+ 4836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.129:50217: BlockingRecv error: recv got EOF from 127.20.81.129:50217 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 10 00:58:53 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:53 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:53.827302 31342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:53.814362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.130:55093)
0810 00:58:53.814873 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:53.814890 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:53.815003 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:53.815516 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:53.815519 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:53.815930 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:53.816081 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:53.816089 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:53.817058 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:53.817062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:53.817999 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:53.818010 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:53.818698 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:53.818702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:53.818829 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:53.823119 (+ 4290us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:53.823143 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:53.827071 (+ 3928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":258,"threads_started":1}
W20260810 00:58:53.827659 31343 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:53.814504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:55093 (local address 127.20.81.190:44873)
0810 00:58:53.815262 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:53.815267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:53.815281 (+ 14us) server_negotiation.cc:407] Connection header received
0810 00:58:53.815325 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:53.815328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:53.815384 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:53.815470 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:53.816207 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:53.816934 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:53.818131 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:53.818568 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:53.818662 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:53.827333 (+ 8671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.130:55093: BlockingRecv error: recv got EOF from 127.20.81.130:55093 (error 108)
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":533,"threads_started":1}
Aug 10 00:58:54 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:54 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:54.026086 31344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:54.015553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.131:52051)
0810 00:58:54.015990 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:54.016006 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:54.016170 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:54.016553 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:54.016558 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:54.017012 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:54.017176 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:54.017186 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:54.018832 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:54.018838 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:54.020385 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:54.020402 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:54.021420 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:54.021424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:54.021546 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:54.022092 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:54.022110 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:54.025749 (+ 3639us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":416,"threads_started":1}
W20260810 00:58:54.026222 31343 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:54.015626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:52051 (local address 127.20.81.190:44873)
0810 00:58:54.015837 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:54.015844 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:54.016098 (+ 254us) server_negotiation.cc:407] Connection header received
0810 00:58:54.016305 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:54.016311 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:54.016413 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:54.016525 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:54.017321 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:54.018662 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:54.020550 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:54.021277 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:54.021387 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:54.025979 (+ 4592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.131:52051: BlockingRecv error: recv got EOF from 127.20.81.131:52051 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 10 00:58:54 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:54 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:54.275581 31345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:54.265365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.129:49193)
0810 00:58:54.265963 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:54.265987 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:54.266158 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:54.266568 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:54.266573 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:54.267094 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:54.267331 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:54.267343 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:54.268852 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:54.268858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:54.270063 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:54.270076 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:54.270878 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:54.270884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:54.271058 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:54.271715 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:54.271733 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:54.275204 (+ 3471us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":281,"threads_started":1}
W20260810 00:58:54.275657 31343 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:54.265481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:49193 (local address 127.20.81.190:44873)
0810 00:58:54.265693 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:54.265700 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:54.266092 (+ 392us) server_negotiation.cc:407] Connection header received
0810 00:58:54.266295 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:54.266300 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:54.266389 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:54.266533 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:54.267496 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:54.268695 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:54.270200 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:54.270712 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:54.270843 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:54.275395 (+ 4552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.129:49193: BlockingRecv error: recv got EOF from 127.20.81.129:49193 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 10 00:58:54 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:54 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:54.838167 31346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:54.828283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.130:59329)
0810 00:58:54.828880 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:54.828912 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:54.829019 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:54.829510 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:54.829513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:54.829954 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:54.830142 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:54.830151 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:54.831296 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:54.831302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:54.832716 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:54.832732 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:54.833615 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:54.833626 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:54.833749 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:54.834317 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:54.834335 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:54.837789 (+ 3454us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":244,"threads_started":1}
W20260810 00:58:54.838470 31349 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:54.828378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:59329 (local address 127.20.81.190:44873)
0810 00:58:54.829217 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:54.829221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:54.829236 (+ 15us) server_negotiation.cc:407] Connection header received
0810 00:58:54.829296 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:54.829300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:54.829368 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:54.829463 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:54.830273 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:54.831039 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:54.832864 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:54.833451 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:54.833557 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:54.837998 (+ 4441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.130:59329: BlockingRecv error: recv got EOF from 127.20.81.130:59329 (error 108)
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":579,"threads_started":1}
Aug 10 00:58:55 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:55 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:55.036629 31351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:55.027018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.131:47225)
0810 00:58:55.027675 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:55.027692 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:55.027822 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:55.028168 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:55.028171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:55.028724 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:55.028930 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:55.028940 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:55.029946 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:55.029950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:55.030956 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:55.030968 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:55.031660 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:55.031664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:55.031778 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:55.032367 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:55.032384 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:55.036265 (+ 3881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":331,"threads_started":1}
W20260810 00:58:55.036728 31349 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:55.027136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:47225 (local address 127.20.81.190:44873)
0810 00:58:55.027408 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:55.027427 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:55.027771 (+ 344us) server_negotiation.cc:407] Connection header received
0810 00:58:55.027920 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:55.027925 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:55.027991 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:55.028080 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:55.029057 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:55.029817 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:55.031109 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:55.031526 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:55.031619 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:55.036493 (+ 4874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.131:47225: BlockingRecv error: recv got EOF from 127.20.81.131:47225 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 10 00:58:55 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:55 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:55.286720 31352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:55.276757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.129:35433)
0810 00:58:55.277346 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:55.277368 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:55.277540 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:55.277945 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:55.277951 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:55.278641 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:55.278860 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:55.278871 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:55.280168 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:55.280175 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:55.281292 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:55.281307 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:55.282021 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:55.282026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:55.282157 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:55.282806 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:55.282825 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:55.286334 (+ 3509us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":264,"threads_started":1}
W20260810 00:58:55.286844 31349 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:55.277003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:35433 (local address 127.20.81.190:44873)
0810 00:58:55.277237 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:55.277244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:55.277458 (+ 214us) server_negotiation.cc:407] Connection header received
0810 00:58:55.277676 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:55.277684 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:55.277773 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:55.277909 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:55.279039 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:55.279978 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:55.281431 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:55.281847 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:55.281961 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:55.286569 (+ 4608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.129:35433: BlockingRecv error: recv got EOF from 127.20.81.129:35433 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 10 00:58:55 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:55 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:55.848750 31353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:55.839162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.130:33155)
0810 00:58:55.839670 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:55.839687 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:55.839816 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:55.840329 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:55.840334 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:55.840799 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:55.840980 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:55.840990 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:55.842189 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:55.842196 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:55.843180 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:55.843191 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:55.843951 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:55.843957 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:55.844131 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:55.844787 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:55.844805 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:55.848505 (+ 3700us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":269,"threads_started":1}
W20260810 00:58:55.848970 31354 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:55.839282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:33155 (local address 127.20.81.190:44873)
0810 00:58:55.840032 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:55.840037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:55.840052 (+ 15us) server_negotiation.cc:407] Connection header received
0810 00:58:55.840096 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:55.840100 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:55.840165 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:55.840300 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:55.841134 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:55.842049 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:55.843321 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:55.843757 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:55.843901 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:55.848718 (+ 4817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.130:33155: BlockingRecv error: recv got EOF from 127.20.81.130:33155 (error 108)
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":524,"threads_started":1}
Aug 10 00:58:56 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:56 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:56.047042 31355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:56.037615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.131:42783)
0810 00:58:56.038168 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:56.038185 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:56.038302 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:56.038613 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:56.038616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:56.039023 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:56.039195 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:56.039203 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:56.040244 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:56.040248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:56.041203 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:56.041215 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:56.041874 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:56.041878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:56.041994 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:56.042579 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:56.042596 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:56.046651 (+ 4055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":296,"threads_started":1}
W20260810 00:58:56.047135 31354 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:56.037709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:42783 (local address 127.20.81.190:44873)
0810 00:58:56.037876 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:56.037880 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:56.038260 (+ 380us) server_negotiation.cc:407] Connection header received
0810 00:58:56.038394 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:56.038416 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:56.038481 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:56.038571 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:56.039313 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:56.040118 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:56.041334 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:56.041721 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:56.041813 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:56.046879 (+ 5066us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.131:42783: BlockingRecv error: recv got EOF from 127.20.81.131:42783 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 10 00:58:56 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:56 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:56.299147 31356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:56.287814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.129:53005)
0810 00:58:56.288478 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:56.288502 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:56.288655 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:56.289189 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:56.289193 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:56.289637 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:56.289785 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:56.289795 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:56.290928 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:56.290933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:56.291891 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:56.291902 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:56.292661 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:56.292666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:56.292853 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:56.293627 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:56.293655 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:56.298677 (+ 5022us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":356,"threads_started":1}
W20260810 00:58:56.302299 31354 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:56.287824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:53005 (local address 127.20.81.190:44873)
0810 00:58:56.288021 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:56.288028 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:56.288598 (+ 570us) server_negotiation.cc:407] Connection header received
0810 00:58:56.288806 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:56.288812 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:56.288935 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:56.289004 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:56.289938 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:56.290773 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:56.292034 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:56.292499 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:56.292645 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:56.302006 (+ 9361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.129:53005: BlockingRecv error: recv got EOF from 127.20.81.129:53005 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 10 00:58:56 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:56 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:56.859709 31357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:56.849743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.130:35073)
0810 00:58:56.850369 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:56.850388 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:56.850558 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:56.850967 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:56.850971 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:56.851409 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:56.851565 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:56.851574 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:56.852627 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:56.852631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:56.853796 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:56.853810 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:56.854603 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:56.854611 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:56.854766 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:56.855502 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:56.855520 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:56.859485 (+ 3965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":322,"threads_started":1}
W20260810 00:58:56.860069 31359 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:56.849817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:35073 (local address 127.20.81.190:44873)
0810 00:58:56.850679 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:56.850683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:56.850699 (+ 16us) server_negotiation.cc:407] Connection header received
0810 00:58:56.850743 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:56.850747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:56.850803 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:56.850893 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:56.851706 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:56.852479 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:56.853955 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:56.854377 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:56.854549 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:56.859726 (+ 5177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.130:35073: BlockingRecv error: recv got EOF from 127.20.81.130:35073 (error 108)
Metrics: {"server-negotiator.queue_time_us":761,"thread_start_us":627,"threads_started":1}
Aug 10 00:58:57 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:57 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:57.057410 31360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:57.048070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.131:49353)
0810 00:58:57.048616 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:57.048638 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:57.048779 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:57.049098 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:57.049101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:57.049563 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:57.049716 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:57.049726 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:57.050795 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:57.050800 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:57.051814 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:57.051827 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:57.052515 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:57.052520 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:57.052656 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:57.053315 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:57.053333 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:57.057146 (+ 3813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":253,"threads_started":1}
W20260810 00:58:57.057511 31359 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:57.048099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:49353 (local address 127.20.81.190:44873)
0810 00:58:57.048325 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:57.048335 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:57.048739 (+ 404us) server_negotiation.cc:407] Connection header received
0810 00:58:57.048904 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:57.048908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:57.048971 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:57.049054 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:57.049844 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:57.050656 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:57.051954 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:57.052372 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:57.052466 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:57.057313 (+ 4847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.131:49353: BlockingRecv error: recv got EOF from 127.20.81.131:49353 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 10 00:58:57 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:57 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:57.314790 31361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:57.300230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.129:34307)
0810 00:58:57.300777 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:57.300796 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:57.300949 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:57.301308 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:57.301315 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:57.301748 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:57.301895 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:57.301903 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:57.303366 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:57.303372 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:57.304459 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:57.304472 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:57.305198 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:57.305204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:57.305391 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:57.311062 (+ 5671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:57.311082 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:57.314551 (+ 3469us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":306,"threads_started":1}
W20260810 00:58:57.314993 31359 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:57.300317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:34307 (local address 127.20.81.190:44873)
0810 00:58:57.300552 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:57.300561 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:57.300875 (+ 314us) server_negotiation.cc:407] Connection header received
0810 00:58:57.301070 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:57.301078 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:57.301154 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:57.301259 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:57.302047 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:57.303214 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:57.304639 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:57.305035 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:57.305166 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:57.314729 (+ 9563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.129:34307: BlockingRecv error: recv got EOF from 127.20.81.129:34307 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 10 00:58:57 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:57 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:57.870680 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:57.860834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.130:59645)
0810 00:58:57.861432 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:57.861452 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:57.861568 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:57.861904 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:57.861908 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:57.862323 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:57.862534 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:57.862544 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:57.863623 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:57.863628 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:57.864757 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:57.864771 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:57.865440 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:57.865444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:57.865574 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:57.866167 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:57.866185 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:57.870354 (+ 4169us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":280,"threads_started":1}
W20260810 00:58:57.870895 31363 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:57.860911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:59645 (local address 127.20.81.190:44873)
0810 00:58:57.861475 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:57.861480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:57.861501 (+ 21us) server_negotiation.cc:407] Connection header received
0810 00:58:57.861685 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:57.861688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:57.861756 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:57.861844 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:57.862670 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:57.863409 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:57.864896 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:57.865302 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:57.865396 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:57.870614 (+ 5218us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.130:59645: BlockingRecv error: recv got EOF from 127.20.81.130:59645 (error 108)
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":296,"threads_started":1}
Aug 10 00:58:58 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:58 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:58.068253 31364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:58.058492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.131:58439)
0810 00:58:58.059306 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:58.059322 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:58.059435 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:58.059729 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:58.059732 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:58.060127 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:58.060293 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:58.060338 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:58.061412 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:58.061416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:58.062899 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:58.062917 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:58.063834 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:58.063839 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:58.063952 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:58.064517 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:58.064535 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:58.067999 (+ 3464us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":550,"threads_started":1}
W20260810 00:58:58.068435 31363 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:58.058704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:58439 (local address 127.20.81.190:44873)
0810 00:58:58.058938 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:58.058948 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:58.059402 (+ 454us) server_negotiation.cc:407] Connection header received
0810 00:58:58.059535 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:58.059539 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:58.059604 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:58.059690 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:58.060480 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:58.061253 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:58.063103 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:58.063692 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:58.063795 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:58.068179 (+ 4384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.131:58439: BlockingRecv error: recv got EOF from 127.20.81.131:58439 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 10 00:58:58 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:58 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:58.326049 31365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:58.315737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.129:39453)
0810 00:58:58.316303 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:58.316321 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:58.316505 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:58.316914 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:58.316919 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:58.317417 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:58.317619 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:58.317629 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:58.318864 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:58.318870 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:58.319896 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:58.319908 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:58.320689 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:58.320696 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:58.320910 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:58.321634 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:58.321654 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:58.325724 (+ 4070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":300,"threads_started":1}
W20260810 00:58:58.326365 31363 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:58.315802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:39453 (local address 127.20.81.190:44873)
0810 00:58:58.316099 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:58.316107 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:58.316433 (+ 326us) server_negotiation.cc:407] Connection header received
0810 00:58:58.316644 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:58.316651 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:58.316747 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:58.316873 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:58.317783 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:58.318702 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:58.320063 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:58.320519 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:58.320669 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:58.326063 (+ 5394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.129:39453: BlockingRecv error: recv got EOF from 127.20.81.129:39453 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
Aug 10 00:58:58 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:58 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:58.881880 31366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:58.871983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.130:59189)
0810 00:58:58.872557 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:58.872581 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:58.872710 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:58.873090 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:58.873093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:58.873541 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:58.873746 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:58.873765 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:58.874948 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:58.874953 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:58.876025 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:58.876038 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:58.876817 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:58.876823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:58.876957 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:58.877605 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:58.877624 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:58.881544 (+ 3920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":306,"threads_started":1}
W20260810 00:58:58.882037 31367 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:58.872084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:59189 (local address 127.20.81.190:44873)
0810 00:58:58.872566 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:58.872570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:58.872642 (+ 72us) server_negotiation.cc:407] Connection header received
0810 00:58:58.872824 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:58.872837 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:58.872913 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:58.873006 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:58.873937 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:58.874805 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:58.876160 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:58.876657 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:58.876768 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:58.881730 (+ 4962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.130:59189: BlockingRecv error: recv got EOF from 127.20.81.130:59189 (error 108)
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":262,"threads_started":1}
Aug 10 00:58:59 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:59 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:59.079293 31368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:59.069200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.131:58085)
0810 00:58:59.069780 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:59.069798 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:59.069918 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:59.070267 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:59.070272 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:59.070776 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:59.071002 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:59.071014 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:59.072078 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:59.072083 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:59.073140 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:59.073153 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:59.073926 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:59.073932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:59.074077 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:59.074891 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:59.074911 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:59.078947 (+ 4036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":350,"threads_started":1}
W20260810 00:58:59.079401 31367 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:59.069560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:58085 (local address 127.20.81.190:44873)
0810 00:58:59.069737 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:59.069742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:59.069879 (+ 137us) server_negotiation.cc:407] Connection header received
0810 00:58:59.070018 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:59.070022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:59.070086 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:59.070175 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:59.071168 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:59.071939 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:59.073299 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:59.073726 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:59.073835 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:59.079151 (+ 5316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.131:58085: BlockingRecv error: recv got EOF from 127.20.81.131:58085 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 10 00:58:59 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:59 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:59.336197 31369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:59.327014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.129:37335)
0810 00:58:59.327585 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:59.327602 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:59.327712 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:59.328078 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:59.328081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:59.328479 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:59.328656 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:59.328664 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:59.329692 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:59.329696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:59.330637 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:59.330648 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:59.331504 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:59.331536 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:59.331734 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:59.332462 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:59.332494 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:59.335950 (+ 3456us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":326,"threads_started":1}
W20260810 00:58:59.336419 31367 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:59.327042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:37335 (local address 127.20.81.190:44873)
0810 00:58:59.327203 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:59.327208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:59.327672 (+ 464us) server_negotiation.cc:407] Connection header received
0810 00:58:59.327808 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:59.327811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:59.327959 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:59.328047 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:59.328776 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:59.329565 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:59.330779 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:59.331374 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:59.331509 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:59.336159 (+ 4650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.129:37335: BlockingRecv error: recv got EOF from 127.20.81.129:37335 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 00:58:59 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:58:59 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:58:59.893033 31370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:58:59.883171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.130:54519)
0810 00:58:59.883999 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0810 00:58:59.884017 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:58:59.884185 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:58:59.884557 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:58:59.884562 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:58:59.885014 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:58:59.885188 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:59.885197 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:59.886248 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:59.886254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:59.887396 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:58:59.887408 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:58:59.888150 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:58:59.888155 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:58:59.888352 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:59.888997 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:58:59.889014 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:58:59.892674 (+ 3660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":575,"threads_started":1}
W20260810 00:58:59.893184 31371 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:58:59.883236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:54519 (local address 127.20.81.190:44873)
0810 00:58:59.883783 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0810 00:58:59.883789 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:58:59.884113 (+ 324us) server_negotiation.cc:407] Connection header received
0810 00:58:59.884313 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:58:59.884320 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:58:59.884400 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:58:59.884523 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:58:59.885338 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:59.886106 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:59.887538 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:58:59.887998 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:58:59.888138 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:58:59.892894 (+ 4756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.130:54519: BlockingRecv error: recv got EOF from 127.20.81.130:54519 (error 108)
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":274,"threads_started":1}
Aug 10 00:59:00 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:59:00 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:59:00.089983 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:00.080317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.131:36007)
0810 00:59:00.080914 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0810 00:59:00.080931 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:59:00.081052 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:59:00.081403 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:59:00.081407 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:59:00.081868 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:59:00.082043 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:00.082053 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:00.083176 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:00.083180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:00.084136 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:00.084148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:00.084851 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:00.084856 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:00.084995 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:00.085602 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:59:00.085626 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:59:00.089730 (+ 4104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":337,"threads_started":1}
W20260810 00:59:00.090188 31371 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:59:00.080487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:36007 (local address 127.20.81.190:44873)
0810 00:59:00.080665 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:00.080670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:00.081008 (+ 338us) server_negotiation.cc:407] Connection header received
0810 00:59:00.081151 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:00.081155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:00.081220 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:00.081309 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:00.082184 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:00.083038 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:00.084257 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:00.084644 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:00.084736 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:00.089929 (+ 5193us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.131:36007: BlockingRecv error: recv got EOF from 127.20.81.131:36007 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 10 00:59:00 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:59:00 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:59:00.348824 31373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:00.337186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.129:38795)
0810 00:59:00.337718 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0810 00:59:00.337734 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:59:00.337912 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:59:00.338313 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:59:00.338318 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:59:00.338917 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:59:00.339152 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:00.339164 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:00.340607 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:00.340613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:00.342059 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:00.342072 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:00.342923 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:00.342929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:00.343136 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:00.343842 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:59:00.343862 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:59:00.348453 (+ 4591us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":284,"threads_started":1}
W20260810 00:59:00.348949 31371 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:59:00.337304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:38795 (local address 127.20.81.190:44873)
0810 00:59:00.337553 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:00.337560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:00.337831 (+ 271us) server_negotiation.cc:407] Connection header received
0810 00:59:00.338053 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:00.338059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:00.338155 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:00.338272 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:00.339320 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:00.340445 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:00.342218 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:00.342753 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:00.342907 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:00.348644 (+ 5737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.129:38795: BlockingRecv error: recv got EOF from 127.20.81.129:38795 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 10 00:59:00 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:59:00 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.130@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:59:00.904299 31374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:00.894070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.130:51297)
0810 00:59:00.894746 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0810 00:59:00.894768 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 00:59:00.894903 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:59:00.895256 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:59:00.895260 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:59:00.895700 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:59:00.895849 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:00.895857 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:00.896995 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:00.897003 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:00.898017 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:00.898029 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:00.898861 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:00.898866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:00.899069 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:00.899757 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:59:00.899776 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:59:00.903830 (+ 4054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":380,"threads_started":1}
W20260810 00:59:00.904443 31375 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:59:00.894258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:51297 (local address 127.20.81.190:44873)
0810 00:59:00.894842 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:00.894849 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:00.894866 (+ 17us) server_negotiation.cc:407] Connection header received
0810 00:59:00.895051 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:00.895055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:00.895117 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:00.895227 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:00.896010 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:00.896846 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:00.898157 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:00.898706 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:00.898836 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:00.904137 (+ 5301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.130:51297: BlockingRecv error: recv got EOF from 127.20.81.130:51297 (error 108)
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":324,"threads_started":1}
Aug 10 00:59:01 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:59:01 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.131@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:59:01.100262 31377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:01.091003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.131:54983)
0810 00:59:01.091495 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0810 00:59:01.091520 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 00:59:01.091635 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:59:01.091991 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:59:01.091998 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:59:01.092406 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:59:01.092564 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:01.092574 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:01.093728 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:01.093733 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:01.094693 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:01.094706 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:01.095409 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:01.095414 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:01.095590 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:01.096234 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:59:01.096251 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:59:01.099975 (+ 3724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":281,"threads_started":1}
W20260810 00:59:01.100466 31375 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:59:01.091117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:54983 (local address 127.20.81.190:44873)
0810 00:59:01.091288 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:01.091293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:01.091596 (+ 303us) server_negotiation.cc:407] Connection header received
0810 00:59:01.091768 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:01.091771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:01.091835 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:01.091934 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:01.092721 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:01.093568 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:01.094841 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:01.095258 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:01.095394 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:01.100213 (+ 4819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.131:54983: BlockingRecv error: recv got EOF from 127.20.81.131:54983 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 00:59:01 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:59:01 dist-test-slave-gp6n krb5kdc[30196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.81.129@KRBTEST.COM for oryx/127.20.81.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:59:01.360457 31378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:01.350257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44873 (local address 127.20.81.129:40847)
0810 00:59:01.350869 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0810 00:59:01.350887 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:59:01.351068 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:59:01.351447 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:59:01.351455 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:59:01.351930 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:59:01.352123 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:01.352133 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:01.353321 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:01.353327 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:01.354372 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:01.354384 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:01.355257 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:01.355262 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:01.355455 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:01.356121 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:59:01.356140 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:59:01.360171 (+ 4031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.81.190:44873: Server oryx/127.20.81.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":323,"threads_started":1}
W20260810 00:59:01.360632 31375 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:59:01.350358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:40847 (local address 127.20.81.190:44873)
0810 00:59:01.350722 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:01.350730 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:01.350987 (+ 257us) server_negotiation.cc:407] Connection header received
0810 00:59:01.351206 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:01.351211 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:01.351294 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:01.351404 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:01.352292 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:01.353154 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:01.354591 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:01.355112 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:01.355217 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:01.360381 (+ 5164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.81.129:40847: BlockingRecv error: recv got EOF from 127.20.81.129:40847 (error 108)
Metrics: {"server-negotiator.queue_time_us":200}
I20260810 00:59:01.820847 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 30850
I20260810 00:59:01.835192 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:01.822250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54650 (local address 127.20.81.129:35173)
0810 00:59:01.823027 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:01.823035 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:01.823067 (+ 32us) server_negotiation.cc:407] Connection header received
0810 00:59:01.823134 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:01.823140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:01.823389 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:01.823564 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:01.825093 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:01.826249 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:01.827722 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:01.828480 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:01.828589 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:01.831559 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:01.831569 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:01.831583 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:59:01.831631 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:01.834594 (+ 2963us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.81.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 00:59:01.834912 (+ 318us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54650: Request ticket server kudu/127.20.81.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":403,"threads_started":1}
W20260810 00:59:01.835309 31380 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54650: Request ticket server kudu/127.20.81.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 00:59:01.835418 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 30850: Not authorized: Client connection negotiation failed: client connection to 127.20.81.129:35173: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.81.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 00:59:01.835651 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 30850
I20260810 00:59:01.858594 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 31015
I20260810 00:59:01.871757 31381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:01.859812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39080 (local address 127.20.81.130:34335)
0810 00:59:01.860539 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:01.860548 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:01.860584 (+ 36us) server_negotiation.cc:407] Connection header received
0810 00:59:01.860739 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:01.860749 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:01.861075 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:01.861305 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:01.862620 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:01.863785 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:01.864954 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:01.865488 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:01.865590 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:01.868190 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:01.868198 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:01.868210 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:59:01.868250 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:01.871186 (+ 2936us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.81.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 00:59:01.871452 (+ 266us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39080: Request ticket server kudu/127.20.81.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":357,"threads_started":1}
W20260810 00:59:01.871793 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 31015: Not authorized: Client connection negotiation failed: client connection to 127.20.81.130:34335: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.81.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 00:59:01.871871 31381 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39080: Request ticket server kudu/127.20.81.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 00:59:01.871984 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 31015
I20260810 00:59:01.894227 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 31168
I20260810 00:59:01.907575 31383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:01.895300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35016 (local address 127.20.81.131:42859)
0810 00:59:01.895829 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:01.895837 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:01.895870 (+ 33us) server_negotiation.cc:407] Connection header received
0810 00:59:01.895938 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:01.895945 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:01.896168 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:01.896335 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:01.898356 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:01.899619 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:01.900927 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:01.901490 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:01.901597 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:01.904171 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:01.904179 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:01.904192 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:59:01.904233 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:01.907005 (+ 2772us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.81.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 00:59:01.907278 (+ 273us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35016: Request ticket server kudu/127.20.81.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":237,"threads_started":1}
W20260810 00:59:01.907686 31383 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35016: Request ticket server kudu/127.20.81.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 00:59:01.907685 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 31168: Not authorized: Client connection negotiation failed: client connection to 127.20.81.131:42859: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.81.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 00:59:01.907850 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 31168
I20260810 00:59:01.927115 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 30772
I20260810 00:59:01.928654 30842 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:59:02.051697 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 30772
2026-08-10T00:59:02Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21837 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T00:59:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:59:02Z Disabled control of system clock
I20260810 00:59:02.148734 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:38237
--webserver_interface=127.20.81.190
--webserver_port=0
--builtin_ntp_servers=127.20.81.148:45467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:38237
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 00:59:02.488281 31388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:59:02.488585 31388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:59:02.488727 31388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:59:02.496166 31388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:59:02.496273 31388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:59:02.496359 31388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:59:02.496474 31388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:59:02.496555 31388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:59:02.505767 31388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:45467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:38237
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:38237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:59:02.507174 31388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:59:02.509052 31388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:59:02.521448 31393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:02.521392 31396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:02.521471 31394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:59:02.521852 31388 server_base.cc:1034] running on GCE node
I20260810 00:59:02.522881 31388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:59:02.524255 31388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:59:02.525502 31388 hybrid_clock.cc:648] HybridClock initialized: now 1786323542525465 us; error 47 us; skew 500 ppm
I20260810 00:59:02.525841 31388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:59:02.531996 31388 webserver.cc:460] Webserver started at http://127.20.81.190:39763/ using document root <none> and password file <none>
I20260810 00:59:02.532733 31388 fs_manager.cc:359] Metadata directory not provided
I20260810 00:59:02.532876 31388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:59:02.533231 31388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:59:02.536193 31388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "71ea61361b8943e9a8eeca7f5a1f9ca4"
format_stamp: "Formatted at 2026-08-10 00:59:02 on dist-test-slave-gp6n"
server_key: "f875f1ffceb881402b822100fd54a89d"
server_key_iv: "44dd62598009f64d099e44abe77ad6b5"
server_key_version: "encryptionkey@0"
I20260810 00:59:02.537067 31388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "71ea61361b8943e9a8eeca7f5a1f9ca4"
format_stamp: "Formatted at 2026-08-10 00:59:02 on dist-test-slave-gp6n"
server_key: "f875f1ffceb881402b822100fd54a89d"
server_key_iv: "44dd62598009f64d099e44abe77ad6b5"
server_key_version: "encryptionkey@0"
I20260810 00:59:02.542064 31388 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260810 00:59:02.545524 31402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:02.546919 31388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260810 00:59:02.547116 31388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "71ea61361b8943e9a8eeca7f5a1f9ca4"
format_stamp: "Formatted at 2026-08-10 00:59:02 on dist-test-slave-gp6n"
server_key: "f875f1ffceb881402b822100fd54a89d"
server_key_iv: "44dd62598009f64d099e44abe77ad6b5"
server_key_version: "encryptionkey@0"
I20260810 00:59:02.547363 31388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:59:02.565266 31388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:59:02.566093 31388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:59:02.566386 31388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:59:02.585777 31388 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:38237
I20260810 00:59:02.585803 31460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:38237 every 8 connection(s)
I20260810 00:59:02.587160 31388 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:59:02.591251 31461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:59:02.592962 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 31388
I20260810 00:59:02.593227 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 00:59:02.594148 20806 external_mini_cluster.cc:1428] Setting key d25fdbd5e492ab6a01a80b2ad77e82b7
I20260810 00:59:02.605197 31461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4: Bootstrap starting.
I20260810 00:59:02.608112 31465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:02.596647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36150 (local address 127.20.81.190:38237)
0810 00:59:02.597701 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:02.597712 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:02.597757 (+ 45us) server_negotiation.cc:407] Connection header received
0810 00:59:02.599427 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:02.599448 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:02.599896 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:02.600289 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:02.600927 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:02.602186 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:02.603566 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:02.604229 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:02.604460 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:02.605795 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:02.605803 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:02.605816 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:59:02.606358 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:02.606510 (+ 152us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:59:02.606534 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:02.606682 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:02.606901 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 00:59:02.607201 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":380,"threads_started":1}
I20260810 00:59:02.608496 31461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4: Neither blocks nor log segments found. Creating new log.
I20260810 00:59:02.609570 31461 log.cc:824] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4: Log is configured to *not* fsync() on all Append() calls
I20260810 00:59:02.612288 31461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4: No bootstrap required, opened a new log
I20260810 00:59:02.619385 31461 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ea61361b8943e9a8eeca7f5a1f9ca4" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38237 } }
I20260810 00:59:02.619768 31461 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:59:02.619907 31461 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71ea61361b8943e9a8eeca7f5a1f9ca4, State: Initialized, Role: FOLLOWER
I20260810 00:59:02.620584 31461 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ea61361b8943e9a8eeca7f5a1f9ca4" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38237 } }
I20260810 00:59:02.620805 31461 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:59:02.620908 31461 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:59:02.621033 31461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:59:02.623157 31461 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ea61361b8943e9a8eeca7f5a1f9ca4" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38237 } }
I20260810 00:59:02.623591 31461 leader_election.cc:302] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71ea61361b8943e9a8eeca7f5a1f9ca4; no voters:
I20260810 00:59:02.624145 31461 leader_election.cc:288] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:59:02.624308 31467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:59:02.625072 31467 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4 [term 1 LEADER]: Becoming Leader. State: Replica: 71ea61361b8943e9a8eeca7f5a1f9ca4, State: Running, Role: LEADER
I20260810 00:59:02.625609 31467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ea61361b8943e9a8eeca7f5a1f9ca4" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38237 } }
I20260810 00:59:02.626256 31461 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:59:02.628508 31469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71ea61361b8943e9a8eeca7f5a1f9ca4. Latest consensus state: current_term: 1 leader_uuid: "71ea61361b8943e9a8eeca7f5a1f9ca4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ea61361b8943e9a8eeca7f5a1f9ca4" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38237 } } }
I20260810 00:59:02.628645 31468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71ea61361b8943e9a8eeca7f5a1f9ca4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ea61361b8943e9a8eeca7f5a1f9ca4" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38237 } } }
I20260810 00:59:02.629195 31469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4 [sys.catalog]: This master's current role is: LEADER
I20260810 00:59:02.629213 31468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4 [sys.catalog]: This master's current role is: LEADER
I20260810 00:59:02.630252 31473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:59:02.636300 31473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:59:02.644892 31473 catalog_manager.cc:1347] Generated new cluster ID: 694a20d8d0784d70984ce9a4afbda9cd
I20260810 00:59:02.644999 31473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:59:02.656227 31473 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:59:02.657562 31473 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:59:02.667708 31473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71ea61361b8943e9a8eeca7f5a1f9ca4: Generated new TSK 0
I20260810 00:59:02.668511 31473 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:59:02.679989 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:38237
--builtin_ntp_servers=127.20.81.148:45467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 00:59:03.030025 31488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:59:03.030308 31488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:59:03.030501 31488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:59:03.037828 31488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:59:03.037926 31488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:59:03.038115 31488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 00:59:03.047575 31488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:45467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:38237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:59:03.048934 31488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:59:03.051021 31488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:59:03.065358 31496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:03.065384 31495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:03.066111 31499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:59:03.066294 31488 server_base.cc:1034] running on GCE node
I20260810 00:59:03.066988 31488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:59:03.067977 31488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:59:03.069243 31488 hybrid_clock.cc:648] HybridClock initialized: now 1786323543069190 us; error 67 us; skew 500 ppm
I20260810 00:59:03.069742 31488 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:59:03.072395 31488 webserver.cc:460] Webserver started at http://127.20.81.129:43253/ using document root <none> and password file <none>
I20260810 00:59:03.073153 31488 fs_manager.cc:359] Metadata directory not provided
I20260810 00:59:03.073256 31488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:59:03.073551 31488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:59:03.076701 31488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "6afc2063748849a2a2d29d7d301151ef"
format_stamp: "Formatted at 2026-08-10 00:59:03 on dist-test-slave-gp6n"
server_key: "c668c7d5d8930caf9d236fbf62f3aea1"
server_key_iv: "a71c6686f8e38fcce6185f3f70eeab51"
server_key_version: "encryptionkey@0"
I20260810 00:59:03.077548 31488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "6afc2063748849a2a2d29d7d301151ef"
format_stamp: "Formatted at 2026-08-10 00:59:03 on dist-test-slave-gp6n"
server_key: "c668c7d5d8930caf9d236fbf62f3aea1"
server_key_iv: "a71c6686f8e38fcce6185f3f70eeab51"
server_key_version: "encryptionkey@0"
I20260810 00:59:03.083062 31488 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260810 00:59:03.086819 31507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:03.088217 31488 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 00:59:03.088413 31488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "6afc2063748849a2a2d29d7d301151ef"
format_stamp: "Formatted at 2026-08-10 00:59:03 on dist-test-slave-gp6n"
server_key: "c668c7d5d8930caf9d236fbf62f3aea1"
server_key_iv: "a71c6686f8e38fcce6185f3f70eeab51"
server_key_version: "encryptionkey@0"
I20260810 00:59:03.088654 31488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:59:03.118952 31488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:59:03.119835 31488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:59:03.120148 31488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:59:03.121142 31488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:59:03.122713 31488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:59:03.122843 31488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:03.122977 31488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:59:03.123064 31488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:03.157595 31488 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:39925
I20260810 00:59:03.157681 31637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:39925 every 8 connection(s)
I20260810 00:59:03.158887 31488 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 00:59:03.159965 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 31488
I20260810 00:59:03.160204 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 00:59:03.161041 20806 external_mini_cluster.cc:1428] Setting key ec42edfff2b92685b709459548d9848b
I20260810 00:59:03.164575 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:38237
--builtin_ntp_servers=127.20.81.148:45467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 00:59:03.174664 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:03.161861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:34913 (local address 127.20.81.190:38237)
0810 00:59:03.162537 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:03.162542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:03.163899 (+ 1357us) server_negotiation.cc:407] Connection header received
0810 00:59:03.166828 (+ 2929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:03.166837 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:03.166934 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:03.167026 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:03.168489 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:03.169180 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:03.171715 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:03.172114 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:03.172210 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:03.173850 (+ 1640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:03.173855 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:03.173858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:59:03.173875 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:03.173925 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:59:03.173937 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:03.174008 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:03.174264 (+ 256us) server_negotiation.cc:299] Negotiation successful
0810 00:59:03.174344 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":445,"threads_started":1}
I20260810 00:59:03.176107 31643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:03.162564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38237 (local address 127.20.81.129:34913)
0810 00:59:03.163719 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0810 00:59:03.163778 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 00:59:03.166478 (+ 2700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:59:03.167349 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:59:03.167363 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:59:03.167489 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:59:03.168296 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:03.168323 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:03.169358 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:03.169366 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:03.171570 (+ 2204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:03.171588 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:03.172347 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:03.172358 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:03.172702 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:03.173491 (+ 789us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:59:03.173497 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:59:03.173551 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:59:03.173652 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:59:03.174036 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:59:03.174050 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:59:03.174061 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 00:59:03.174247 (+ 186us) client_negotiation.cc:240] Negotiation successful
0810 00:59:03.174710 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":336,"threads_started":1}
I20260810 00:59:03.177850 31638 heartbeater.cc:347] Connected to a master server at 127.20.81.190:38237
I20260810 00:59:03.178153 31638 heartbeater.cc:464] Registering TS with master...
I20260810 00:59:03.178980 31638 heartbeater.cc:511] Master 127.20.81.190:38237 requested a full tablet report, sending...
I20260810 00:59:03.181784 31423 ts_manager.cc:194] Registered new tserver with Master: 6afc2063748849a2a2d29d7d301151ef (127.20.81.129:39925)
I20260810 00:59:03.184116 31423 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.81.129:34913
W20260810 00:59:03.527961 31644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:59:03.528239 31644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:59:03.528321 31644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:59:03.535851 31644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:59:03.535952 31644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:59:03.536055 31644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 00:59:03.544963 31644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:45467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:38237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:59:03.546253 31644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:59:03.548038 31644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:59:03.561306 31651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:03.561369 31652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:03.561427 31654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:59:03.562444 31644 server_base.cc:1034] running on GCE node
I20260810 00:59:03.563058 31644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:59:03.563995 31644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:59:03.565233 31644 hybrid_clock.cc:648] HybridClock initialized: now 1786323543565202 us; error 53 us; skew 500 ppm
I20260810 00:59:03.565574 31644 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:59:03.568004 31644 webserver.cc:460] Webserver started at http://127.20.81.130:41795/ using document root <none> and password file <none>
I20260810 00:59:03.568709 31644 fs_manager.cc:359] Metadata directory not provided
I20260810 00:59:03.568799 31644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:59:03.569159 31644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:59:03.572227 31644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "e176e50902584cf5b23e50bf6aaa1107"
format_stamp: "Formatted at 2026-08-10 00:59:03 on dist-test-slave-gp6n"
server_key: "7dc312b943cca8bb54ea98a3f67eb79f"
server_key_iv: "095b1285bde41a0e57c70fb15b33d041"
server_key_version: "encryptionkey@0"
I20260810 00:59:03.573026 31644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "e176e50902584cf5b23e50bf6aaa1107"
format_stamp: "Formatted at 2026-08-10 00:59:03 on dist-test-slave-gp6n"
server_key: "7dc312b943cca8bb54ea98a3f67eb79f"
server_key_iv: "095b1285bde41a0e57c70fb15b33d041"
server_key_version: "encryptionkey@0"
I20260810 00:59:03.578186 31644 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 00:59:03.581856 31661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:03.583341 31644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:59:03.583513 31644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "e176e50902584cf5b23e50bf6aaa1107"
format_stamp: "Formatted at 2026-08-10 00:59:03 on dist-test-slave-gp6n"
server_key: "7dc312b943cca8bb54ea98a3f67eb79f"
server_key_iv: "095b1285bde41a0e57c70fb15b33d041"
server_key_version: "encryptionkey@0"
I20260810 00:59:03.583722 31644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:59:03.614216 31644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:59:03.615109 31644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:59:03.615420 31644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:59:03.616369 31644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:59:03.617789 31644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:59:03.617899 31644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:03.618036 31644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:59:03.618129 31644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:03.652590 31644 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:41203
I20260810 00:59:03.652700 31790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:41203 every 8 connection(s)
I20260810 00:59:03.653905 31644 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 00:59:03.658872 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 31644
I20260810 00:59:03.659091 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 00:59:03.659698 20806 external_mini_cluster.cc:1428] Setting key 57e9389369e682917ec0b289dc549db5
I20260810 00:59:03.662957 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:38237
--builtin_ntp_servers=127.20.81.148:45467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 00:59:03.668843 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:03.656729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:34435 (local address 127.20.81.190:38237)
0810 00:59:03.656956 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:03.656963 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:03.658500 (+ 1537us) server_negotiation.cc:407] Connection header received
0810 00:59:03.661019 (+ 2519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:03.661025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:03.661098 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:03.661211 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:03.662614 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:03.663438 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:03.665965 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:03.666358 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:03.666553 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:03.668127 (+ 1574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:03.668140 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:03.668143 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:59:03.668162 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:03.668194 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:59:03.668206 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:03.668278 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:03.668545 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 00:59:03.668624 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 00:59:03.670449 31794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:03.657307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38237 (local address 127.20.81.130:34435)
0810 00:59:03.658256 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0810 00:59:03.658295 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 00:59:03.660723 (+ 2428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:59:03.661441 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:59:03.661450 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:59:03.661567 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:59:03.662394 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:03.662454 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:03.663598 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:03.663607 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:03.665814 (+ 2207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:03.665831 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:03.666653 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:03.666662 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:03.666982 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:03.667739 (+ 757us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:59:03.667744 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:59:03.667800 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:59:03.667904 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:59:03.668308 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:59:03.668320 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:59:03.668338 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 00:59:03.668530 (+ 192us) client_negotiation.cc:240] Negotiation successful
0810 00:59:03.668958 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":308,"threads_started":1}
I20260810 00:59:03.671941 31792 heartbeater.cc:347] Connected to a master server at 127.20.81.190:38237
I20260810 00:59:03.672261 31792 heartbeater.cc:464] Registering TS with master...
I20260810 00:59:03.673207 31792 heartbeater.cc:511] Master 127.20.81.190:38237 requested a full tablet report, sending...
I20260810 00:59:03.674798 31423 ts_manager.cc:194] Registered new tserver with Master: e176e50902584cf5b23e50bf6aaa1107 (127.20.81.130:41203)
I20260810 00:59:03.675757 31423 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.81.130:34435
W20260810 00:59:04.010700 31795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:59:04.010983 31795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:59:04.011070 31795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:59:04.018141 31795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:59:04.018244 31795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:59:04.018353 31795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 00:59:04.032374 31795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:45467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:38237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:59:04.034253 31795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:59:04.037009 31795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:59:04.054968 31805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:04.055004 31803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:04.055327 31802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:59:04.057257 31795 server_base.cc:1034] running on GCE node
I20260810 00:59:04.058112 31795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:59:04.059525 31795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:59:04.060793 31795 hybrid_clock.cc:648] HybridClock initialized: now 1786323544060716 us; error 62 us; skew 500 ppm
I20260810 00:59:04.061245 31795 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:59:04.064401 31795 webserver.cc:460] Webserver started at http://127.20.81.131:38897/ using document root <none> and password file <none>
I20260810 00:59:04.065418 31795 fs_manager.cc:359] Metadata directory not provided
I20260810 00:59:04.065558 31795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:59:04.065989 31795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:59:04.070583 31795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "3fa78befc5134d42b05e32ad222efb6a"
format_stamp: "Formatted at 2026-08-10 00:59:04 on dist-test-slave-gp6n"
server_key: "dd2cada9555e2af312d1622bcfe28a2d"
server_key_iv: "df3f8c303dc5ac41b6ff919b140279ff"
server_key_version: "encryptionkey@0"
I20260810 00:59:04.071743 31795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "3fa78befc5134d42b05e32ad222efb6a"
format_stamp: "Formatted at 2026-08-10 00:59:04 on dist-test-slave-gp6n"
server_key: "dd2cada9555e2af312d1622bcfe28a2d"
server_key_iv: "df3f8c303dc5ac41b6ff919b140279ff"
server_key_version: "encryptionkey@0"
I20260810 00:59:04.078531 31795 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 00:59:04.082003 31811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:04.083557 31795 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:59:04.083685 31795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "3fa78befc5134d42b05e32ad222efb6a"
format_stamp: "Formatted at 2026-08-10 00:59:04 on dist-test-slave-gp6n"
server_key: "dd2cada9555e2af312d1622bcfe28a2d"
server_key_iv: "df3f8c303dc5ac41b6ff919b140279ff"
server_key_version: "encryptionkey@0"
I20260810 00:59:04.083856 31795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:59:04.110625 31795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:59:04.111454 31795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:59:04.111732 31795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:59:04.112677 31795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:59:04.114146 31795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:59:04.114235 31795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:04.114315 31795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:59:04.114372 31795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:04.150071 31795 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:41917
I20260810 00:59:04.150170 31940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:41917 every 8 connection(s)
I20260810 00:59:04.151599 31795 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 00:59:04.154663 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 31795
I20260810 00:59:04.154865 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 00:59:04.155314 20806 external_mini_cluster.cc:1428] Setting key f70687837f7400d938fb4801e5c8a007
I20260810 00:59:04.165165 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:04.154445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:53849 (local address 127.20.81.190:38237)
0810 00:59:04.154670 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:04.154678 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:04.156376 (+ 1698us) server_negotiation.cc:407] Connection header received
0810 00:59:04.158859 (+ 2483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:04.158865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:04.158958 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:04.159065 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:04.160102 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:04.160834 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:04.162397 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:04.162843 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:04.162977 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:04.164337 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:04.164342 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:04.164345 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:59:04.164363 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:04.164396 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:59:04.164409 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:04.164505 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:04.164719 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 00:59:04.164830 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 00:59:04.166096 31944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:04.155095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38237 (local address 127.20.81.131:53849)
0810 00:59:04.156228 (+ 1133us) negotiation.cc:107] Waiting for socket to connect
0810 00:59:04.156277 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 00:59:04.158514 (+ 2237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:59:04.159295 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:59:04.159303 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:59:04.159402 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:59:04.159942 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:04.159958 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:04.160995 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:04.161000 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:04.162247 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:04.162258 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:04.163084 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:04.163092 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:04.163316 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:04.164017 (+ 701us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:59:04.164020 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:59:04.164059 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:59:04.164136 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:59:04.164506 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:59:04.164519 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:59:04.164530 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 00:59:04.164719 (+ 189us) client_negotiation.cc:240] Negotiation successful
0810 00:59:04.165093 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":328,"threads_started":1}
I20260810 00:59:04.167346 31941 heartbeater.cc:347] Connected to a master server at 127.20.81.190:38237
I20260810 00:59:04.167655 31941 heartbeater.cc:464] Registering TS with master...
I20260810 00:59:04.168510 31941 heartbeater.cc:511] Master 127.20.81.190:38237 requested a full tablet report, sending...
I20260810 00:59:04.170157 31422 ts_manager.cc:194] Registered new tserver with Master: 3fa78befc5134d42b05e32ad222efb6a (127.20.81.131:41917)
I20260810 00:59:04.171329 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:59:04.171393 31422 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.81.131:53849
W20260810 00:59:04.178608 31642 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:59:04.176881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36158 (local address 127.20.81.190:38237)
0810 00:59:04.177074 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:04.177079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:04.177552 (+ 473us) server_negotiation.cc:407] Connection header received
0810 00:59:04.177689 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:04.177692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:04.177753 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:04.177845 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:04.178310 (+ 465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36158: BlockingRecv error: recv got EOF from 127.0.0.1:36158 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 00:59:04.178987 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 31488
I20260810 00:59:04.186686 31951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:04.180094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48218 (local address 127.20.81.129:39925)
0810 00:59:04.180658 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:04.180664 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:04.180690 (+ 26us) server_negotiation.cc:407] Connection header received
0810 00:59:04.180747 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:04.180753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:04.180953 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:04.181086 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:04.181424 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:04.182493 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:04.183725 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:04.184269 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:04.184369 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:04.185317 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:04.185324 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:04.185336 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:59:04.185897 (+ 561us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:04.185941 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:59:04.185966 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:04.186051 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:04.186175 (+ 124us) server_negotiation.cc:299] Negotiation successful
0810 00:59:04.186309 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":278,"threads_started":1}
I20260810 00:59:04.187472 31638 heartbeater.cc:503] Master 127.20.81.190:38237 was elected leader, sending a full tablet report...
I20260810 00:59:04.188227 31633 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:59:04.272869 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 31488
I20260810 00:59:04.293619 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 31644
I20260810 00:59:04.301375 31953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:04.294608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46250 (local address 127.20.81.130:41203)
0810 00:59:04.295145 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:04.295153 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:04.295180 (+ 27us) server_negotiation.cc:407] Connection header received
0810 00:59:04.295247 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:04.295253 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:04.295476 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:04.295611 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:04.295987 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:04.296981 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:04.298173 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:04.298800 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:04.298898 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:04.299913 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:04.299920 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:04.299933 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:59:04.300555 (+ 622us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:04.300602 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:59:04.300633 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:04.300722 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:04.300866 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 00:59:04.301010 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":270,"threads_started":1}
I20260810 00:59:04.302762 31786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:59:04.388163 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 31644
I20260810 00:59:04.409133 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 31795
I20260810 00:59:04.417551 31955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:04.410189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47798 (local address 127.20.81.131:41917)
0810 00:59:04.410905 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:04.410915 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:04.410946 (+ 31us) server_negotiation.cc:407] Connection header received
0810 00:59:04.411008 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:04.411014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:04.411228 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:04.411404 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:04.411886 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:04.412938 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:04.414195 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:04.414858 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:04.414999 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:04.416076 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:04.416085 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:04.416099 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:59:04.416691 (+ 592us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:04.416737 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:59:04.416756 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:04.416900 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:04.417050 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 00:59:04.417178 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":420,"threads_started":1}
I20260810 00:59:04.418850 31936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:59:04.500368 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 31795
I20260810 00:59:04.519271 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 31388
I20260810 00:59:04.520514 31456 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:59:04.623956 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 31388
2026-08-10T00:59:04Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2564 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T00:59:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:59:04Z Disabled control of system clock
I20260810 00:59:04.709656 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:44743
--webserver_interface=127.20.81.190
--webserver_port=0
--builtin_ntp_servers=127.20.81.148:41603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:44743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 00:59:05.047317 31961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:59:05.047631 31961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:59:05.047717 31961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:59:05.055243 31961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:59:05.055359 31961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:59:05.055398 31961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:59:05.055461 31961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:59:05.055508 31961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:59:05.064800 31961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:41603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:44743
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:44743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:59:05.066097 31961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:59:05.067940 31961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:59:05.080297 31966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:05.080579 31969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:05.080713 31967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:59:05.081794 31961 server_base.cc:1034] running on GCE node
I20260810 00:59:05.082724 31961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:59:05.084107 31961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:59:05.085314 31961 hybrid_clock.cc:648] HybridClock initialized: now 1786323545085264 us; error 47 us; skew 500 ppm
I20260810 00:59:05.085690 31961 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:59:05.088069 31961 webserver.cc:460] Webserver started at http://127.20.81.190:34581/ using document root <none> and password file <none>
I20260810 00:59:05.088790 31961 fs_manager.cc:359] Metadata directory not provided
I20260810 00:59:05.088913 31961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:59:05.089264 31961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:59:05.092232 31961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "194b6be953364333a3e31c04efbb6bbe"
format_stamp: "Formatted at 2026-08-10 00:59:05 on dist-test-slave-gp6n"
server_key: "addf5bf23bd3c065fe612fc76b1e9e60"
server_key_iv: "7323b3eeabb807b326ba18430aa8ad3b"
server_key_version: "encryptionkey@0"
I20260810 00:59:05.093050 31961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "194b6be953364333a3e31c04efbb6bbe"
format_stamp: "Formatted at 2026-08-10 00:59:05 on dist-test-slave-gp6n"
server_key: "addf5bf23bd3c065fe612fc76b1e9e60"
server_key_iv: "7323b3eeabb807b326ba18430aa8ad3b"
server_key_version: "encryptionkey@0"
I20260810 00:59:05.098111 31961 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 00:59:05.101423 31976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:05.102737 31961 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 00:59:05.102913 31961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "194b6be953364333a3e31c04efbb6bbe"
format_stamp: "Formatted at 2026-08-10 00:59:05 on dist-test-slave-gp6n"
server_key: "addf5bf23bd3c065fe612fc76b1e9e60"
server_key_iv: "7323b3eeabb807b326ba18430aa8ad3b"
server_key_version: "encryptionkey@0"
I20260810 00:59:05.103133 31961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:59:05.108729 31961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:59:05.109506 31961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:59:05.109815 31961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:59:05.127502 31961 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:44743
I20260810 00:59:05.127549 32035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:44743 every 8 connection(s)
I20260810 00:59:05.128875 31961 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:59:05.132712 32037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:59:05.134541 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 31961
I20260810 00:59:05.134735 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 00:59:05.135627 20806 external_mini_cluster.cc:1428] Setting key 87f571d811f9ea4fd44b05ed4134b44a
I20260810 00:59:05.145479 32040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:05.138088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50668 (local address 127.20.81.190:44743)
0810 00:59:05.139722 (+ 1634us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:05.139735 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:05.139776 (+ 41us) server_negotiation.cc:407] Connection header received
0810 00:59:05.141912 (+ 2136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:05.141934 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:05.142355 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:05.142767 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:05.143919 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:05.143926 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:05.143937 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:59:05.143965 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:05.144022 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:59:05.144041 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:05.144231 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:05.144326 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 00:59:05.144595 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1222,"thread_start_us":864,"threads_started":1}
I20260810 00:59:05.146226 32037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe: Bootstrap starting.
I20260810 00:59:05.149724 32037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe: Neither blocks nor log segments found. Creating new log.
I20260810 00:59:05.150835 32037 log.cc:824] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe: Log is configured to *not* fsync() on all Append() calls
I20260810 00:59:05.153610 32037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe: No bootstrap required, opened a new log
I20260810 00:59:05.161136 32037 raft_consensus.cc:348] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "194b6be953364333a3e31c04efbb6bbe" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44743 } }
I20260810 00:59:05.161541 32037 raft_consensus.cc:374] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:59:05.161653 32037 raft_consensus.cc:729] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 194b6be953364333a3e31c04efbb6bbe, State: Initialized, Role: FOLLOWER
I20260810 00:59:05.162211 32037 consensus_queue.cc:260] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "194b6be953364333a3e31c04efbb6bbe" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44743 } }
I20260810 00:59:05.162513 32037 raft_consensus.cc:388] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:59:05.162658 32037 raft_consensus.cc:482] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:59:05.162832 32037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:59:05.165398 32037 raft_consensus.cc:504] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "194b6be953364333a3e31c04efbb6bbe" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44743 } }
I20260810 00:59:05.165900 32037 leader_election.cc:302] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 194b6be953364333a3e31c04efbb6bbe; no voters:
I20260810 00:59:05.166546 32037 leader_election.cc:288] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:59:05.166671 32042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:59:05.167408 32042 raft_consensus.cc:686] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe [term 1 LEADER]: Becoming Leader. State: Replica: 194b6be953364333a3e31c04efbb6bbe, State: Running, Role: LEADER
I20260810 00:59:05.167987 32042 consensus_queue.cc:237] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "194b6be953364333a3e31c04efbb6bbe" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44743 } }
I20260810 00:59:05.168604 32037 sys_catalog.cc:564] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:59:05.171037 32044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 194b6be953364333a3e31c04efbb6bbe. Latest consensus state: current_term: 1 leader_uuid: "194b6be953364333a3e31c04efbb6bbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "194b6be953364333a3e31c04efbb6bbe" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44743 } } }
I20260810 00:59:05.171150 32043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "194b6be953364333a3e31c04efbb6bbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "194b6be953364333a3e31c04efbb6bbe" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44743 } } }
I20260810 00:59:05.171521 32044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe [sys.catalog]: This master's current role is: LEADER
I20260810 00:59:05.171564 32043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe [sys.catalog]: This master's current role is: LEADER
I20260810 00:59:05.173094 32052 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:59:05.178391 32052 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:59:05.187935 32052 catalog_manager.cc:1347] Generated new cluster ID: 0c2f158c370a40cf9ce82adc0e08f545
I20260810 00:59:05.188110 32052 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:59:05.221634 32052 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:59:05.222157 32052 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:59:05.238075 32052 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 194b6be953364333a3e31c04efbb6bbe: Generated new TSK 0
I20260810 00:59:05.239071 32052 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:59:05.253125 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44743
--builtin_ntp_servers=127.20.81.148:41603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 00:59:05.619958 32064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:59:05.620226 32064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:59:05.620313 32064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:59:05.627466 32064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:59:05.627561 32064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:59:05.627667 32064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 00:59:05.636579 32064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:41603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44743
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:59:05.637838 32064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:59:05.639560 32064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:59:05.652865 32072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:05.652845 32070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:05.652962 32069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:59:05.655876 32064 server_base.cc:1034] running on GCE node
I20260810 00:59:05.656556 32064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:59:05.657531 32064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:59:05.658726 32064 hybrid_clock.cc:648] HybridClock initialized: now 1786323545658688 us; error 35 us; skew 500 ppm
I20260810 00:59:05.659045 32064 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:59:05.661466 32064 webserver.cc:460] Webserver started at http://127.20.81.129:32879/ using document root <none> and password file <none>
I20260810 00:59:05.662173 32064 fs_manager.cc:359] Metadata directory not provided
I20260810 00:59:05.662292 32064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:59:05.662683 32064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:59:05.665598 32064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "cc8118107ba44d4db7b44842be015315"
format_stamp: "Formatted at 2026-08-10 00:59:05 on dist-test-slave-gp6n"
server_key: "d3c58dff82f2447f6fb07c1aea657c30"
server_key_iv: "57a738322b9ac519b04cc6ca0ac0f554"
server_key_version: "encryptionkey@0"
I20260810 00:59:05.666472 32064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "cc8118107ba44d4db7b44842be015315"
format_stamp: "Formatted at 2026-08-10 00:59:05 on dist-test-slave-gp6n"
server_key: "d3c58dff82f2447f6fb07c1aea657c30"
server_key_iv: "57a738322b9ac519b04cc6ca0ac0f554"
server_key_version: "encryptionkey@0"
I20260810 00:59:05.672230 32064 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260810 00:59:05.676119 32080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:05.677443 32064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 00:59:05.677618 32064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "cc8118107ba44d4db7b44842be015315"
format_stamp: "Formatted at 2026-08-10 00:59:05 on dist-test-slave-gp6n"
server_key: "d3c58dff82f2447f6fb07c1aea657c30"
server_key_iv: "57a738322b9ac519b04cc6ca0ac0f554"
server_key_version: "encryptionkey@0"
I20260810 00:59:05.677841 32064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:59:05.683735 32064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:59:05.684502 32064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:59:05.684770 32064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:59:05.685672 32064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:59:05.687142 32064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:59:05.687269 32064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:05.687398 32064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:59:05.687490 32064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:05.721225 32064 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:33307
I20260810 00:59:05.721283 32210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:33307 every 8 connection(s)
I20260810 00:59:05.722635 32064 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 00:59:05.727116 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 32064
I20260810 00:59:05.727321 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 00:59:05.728214 20806 external_mini_cluster.cc:1428] Setting key f9efa7d5a8d86e55459a5630c04f561a
I20260810 00:59:05.731714 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44743
--builtin_ntp_servers=127.20.81.148:41603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 00:59:05.732208 32214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:05.725215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:58867 (local address 127.20.81.190:44743)
0810 00:59:05.725649 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:05.725655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:05.727116 (+ 1461us) server_negotiation.cc:407] Connection header received
0810 00:59:05.729545 (+ 2429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:05.729549 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:05.729599 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:05.729694 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:05.731261 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:05.731267 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:05.731271 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:59:05.731291 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:05.731362 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:59:05.731380 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:05.731509 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:05.731803 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 00:59:05.731870 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":216,"threads_started":1}
I20260810 00:59:05.733422 32215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:05.725829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44743 (local address 127.20.81.129:58867)
0810 00:59:05.726918 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0810 00:59:05.726972 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 00:59:05.729223 (+ 2251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:59:05.729971 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:59:05.729982 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:59:05.730105 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:59:05.730860 (+ 755us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:59:05.730863 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:59:05.730924 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:59:05.731042 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:59:05.731567 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:59:05.731580 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:59:05.731588 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 00:59:05.731893 (+ 305us) client_negotiation.cc:240] Negotiation successful
0810 00:59:05.732163 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":281,"threads_started":1}
I20260810 00:59:05.735131 32212 heartbeater.cc:347] Connected to a master server at 127.20.81.190:44743
I20260810 00:59:05.735462 32212 heartbeater.cc:464] Registering TS with master...
I20260810 00:59:05.736143 32212 heartbeater.cc:511] Master 127.20.81.190:44743 requested a full tablet report, sending...
I20260810 00:59:05.738612 31995 ts_manager.cc:194] Registered new tserver with Master: cc8118107ba44d4db7b44842be015315 (127.20.81.129:33307)
W20260810 00:59:06.073813 32216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:59:06.074170 32216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:59:06.074331 32216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:59:06.081668 32216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:59:06.081799 32216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:59:06.081976 32216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 00:59:06.091220 32216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:41603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44743
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:59:06.092583 32216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:59:06.094381 32216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:59:06.107613 32221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:06.107532 32224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:06.107499 32222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:59:06.108559 32216 server_base.cc:1034] running on GCE node
I20260810 00:59:06.109133 32216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:59:06.110133 32216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:59:06.111351 32216 hybrid_clock.cc:648] HybridClock initialized: now 1786323546111293 us; error 51 us; skew 500 ppm
I20260810 00:59:06.111665 32216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:59:06.113930 32216 webserver.cc:460] Webserver started at http://127.20.81.130:41353/ using document root <none> and password file <none>
I20260810 00:59:06.114691 32216 fs_manager.cc:359] Metadata directory not provided
I20260810 00:59:06.114821 32216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:59:06.115144 32216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:59:06.117957 32216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "e75fb8c4bb01407baafa93e0ddf06530"
format_stamp: "Formatted at 2026-08-10 00:59:06 on dist-test-slave-gp6n"
server_key: "c1b28c360bdab4c207a9986f05110df5"
server_key_iv: "84c1e0651692c927aa7d7e008669f63f"
server_key_version: "encryptionkey@0"
I20260810 00:59:06.118818 32216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "e75fb8c4bb01407baafa93e0ddf06530"
format_stamp: "Formatted at 2026-08-10 00:59:06 on dist-test-slave-gp6n"
server_key: "c1b28c360bdab4c207a9986f05110df5"
server_key_iv: "84c1e0651692c927aa7d7e008669f63f"
server_key_version: "encryptionkey@0"
I20260810 00:59:06.124150 32216 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 00:59:06.127625 32231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:06.129005 32216 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 00:59:06.129156 32216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "e75fb8c4bb01407baafa93e0ddf06530"
format_stamp: "Formatted at 2026-08-10 00:59:06 on dist-test-slave-gp6n"
server_key: "c1b28c360bdab4c207a9986f05110df5"
server_key_iv: "84c1e0651692c927aa7d7e008669f63f"
server_key_version: "encryptionkey@0"
I20260810 00:59:06.129379 32216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:59:06.135095 32216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:59:06.135949 32216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:59:06.136269 32216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:59:06.137323 32216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:59:06.138958 32216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:59:06.139055 32216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:06.139220 32216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:59:06.139334 32216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:06.175717 32216 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:40777
I20260810 00:59:06.175770 32356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:40777 every 8 connection(s)
I20260810 00:59:06.177192 32216 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 00:59:06.181399 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 32216
I20260810 00:59:06.181597 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 00:59:06.182197 20806 external_mini_cluster.cc:1428] Setting key eb98a61c21f09ee82d83b2452f3b27df
I20260810 00:59:06.185613 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44743
--builtin_ntp_servers=127.20.81.148:41603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 00:59:06.186290 32214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:06.179842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:43033 (local address 127.20.81.190:44743)
0810 00:59:06.180048 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:06.180052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:06.181670 (+ 1618us) server_negotiation.cc:407] Connection header received
0810 00:59:06.183715 (+ 2045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:06.183721 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:06.183801 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:06.183923 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:06.185413 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:06.185417 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:06.185421 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:59:06.185438 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:06.185474 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:59:06.185486 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:06.185612 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:06.185896 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 00:59:06.185975 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 00:59:06.188156 32359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:06.180396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44743 (local address 127.20.81.130:43033)
0810 00:59:06.181485 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0810 00:59:06.181538 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 00:59:06.183455 (+ 1917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:59:06.184179 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:59:06.184191 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:59:06.184295 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:59:06.185033 (+ 738us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:59:06.185037 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:59:06.185086 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:59:06.185206 (+ 120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:59:06.185609 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:59:06.185620 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:59:06.185630 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 00:59:06.186628 (+ 998us) client_negotiation.cc:240] Negotiation successful
0810 00:59:06.186894 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":271,"threads_started":1}
I20260810 00:59:06.189649 32357 heartbeater.cc:347] Connected to a master server at 127.20.81.190:44743
I20260810 00:59:06.189955 32357 heartbeater.cc:464] Registering TS with master...
I20260810 00:59:06.190605 32357 heartbeater.cc:511] Master 127.20.81.190:44743 requested a full tablet report, sending...
I20260810 00:59:06.192004 31995 ts_manager.cc:194] Registered new tserver with Master: e75fb8c4bb01407baafa93e0ddf06530 (127.20.81.130:40777)
W20260810 00:59:06.531814 32360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:59:06.532140 32360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:59:06.532312 32360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:59:06.539347 32360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:59:06.539480 32360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:59:06.539660 32360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 00:59:06.548550 32360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:41603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44743
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:59:06.549891 32360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:59:06.551744 32360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:59:06.564824 32369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:06.564791 32366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:06.565534 32365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:59:06.565572 32360 server_base.cc:1034] running on GCE node
I20260810 00:59:06.566193 32360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:59:06.567121 32360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:59:06.568332 32360 hybrid_clock.cc:648] HybridClock initialized: now 1786323546568298 us; error 27 us; skew 500 ppm
I20260810 00:59:06.568656 32360 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:59:06.571028 32360 webserver.cc:460] Webserver started at http://127.20.81.131:43909/ using document root <none> and password file <none>
I20260810 00:59:06.571717 32360 fs_manager.cc:359] Metadata directory not provided
I20260810 00:59:06.571839 32360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:59:06.572173 32360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:59:06.575335 32360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "f855407b4073449f90c91196275a1f19"
format_stamp: "Formatted at 2026-08-10 00:59:06 on dist-test-slave-gp6n"
server_key: "a262af9b53569fd48ca10d654e8c94b3"
server_key_iv: "69cda77cdeadbb34a5db37b4cb66194f"
server_key_version: "encryptionkey@0"
I20260810 00:59:06.576113 32360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "f855407b4073449f90c91196275a1f19"
format_stamp: "Formatted at 2026-08-10 00:59:06 on dist-test-slave-gp6n"
server_key: "a262af9b53569fd48ca10d654e8c94b3"
server_key_iv: "69cda77cdeadbb34a5db37b4cb66194f"
server_key_version: "encryptionkey@0"
I20260810 00:59:06.581142 32360 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 00:59:06.585075 32375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:06.586544 32360 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:59:06.586717 32360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "f855407b4073449f90c91196275a1f19"
format_stamp: "Formatted at 2026-08-10 00:59:06 on dist-test-slave-gp6n"
server_key: "a262af9b53569fd48ca10d654e8c94b3"
server_key_iv: "69cda77cdeadbb34a5db37b4cb66194f"
server_key_version: "encryptionkey@0"
I20260810 00:59:06.586930 32360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:59:06.592509 32360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:59:06.593261 32360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:59:06.593538 32360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:59:06.594501 32360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:59:06.595865 32360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:59:06.595980 32360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:06.596114 32360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:59:06.596208 32360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:06.629529 32360 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:41097
I20260810 00:59:06.629622 32503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:41097 every 8 connection(s)
I20260810 00:59:06.630906 32360 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 00:59:06.635337 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 32360
I20260810 00:59:06.635550 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 00:59:06.636099 20806 external_mini_cluster.cc:1428] Setting key 884885b1797cb5fea68b274f64a6be99
I20260810 00:59:06.639886 32214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:06.633599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:54809 (local address 127.20.81.190:44743)
0810 00:59:06.633845 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:06.633854 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:06.635555 (+ 1701us) server_negotiation.cc:407] Connection header received
0810 00:59:06.637374 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:06.637380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:06.637450 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:06.637571 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:06.639084 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:06.639091 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:06.639097 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:59:06.639120 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:06.639169 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:59:06.639187 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:06.639356 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:06.639556 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 00:59:06.639631 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260810 00:59:06.640730 32506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:06.634261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44743 (local address 127.20.81.131:54809)
0810 00:59:06.635377 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0810 00:59:06.635429 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 00:59:06.637103 (+ 1674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:59:06.637806 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:59:06.637815 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:59:06.637905 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:59:06.638705 (+ 800us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:59:06.638709 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:59:06.638758 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:59:06.638871 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:59:06.639305 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:59:06.639318 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:59:06.639327 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 00:59:06.639587 (+ 260us) client_negotiation.cc:240] Negotiation successful
0810 00:59:06.639778 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":408,"threads_started":1}
I20260810 00:59:06.642088 32504 heartbeater.cc:347] Connected to a master server at 127.20.81.190:44743
I20260810 00:59:06.642365 32504 heartbeater.cc:464] Registering TS with master...
I20260810 00:59:06.642988 32504 heartbeater.cc:511] Master 127.20.81.190:44743 requested a full tablet report, sending...
I20260810 00:59:06.644420 31995 ts_manager.cc:194] Registered new tserver with Master: f855407b4073449f90c91196275a1f19 (127.20.81.131:41097)
I20260810 00:59:06.651713 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 00:59:06.657887 32214 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:59:06.656302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50670 (local address 127.20.81.190:44743)
0810 00:59:06.656459 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:06.656480 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:06.656954 (+ 474us) server_negotiation.cc:407] Connection header received
0810 00:59:06.657166 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:06.657170 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:06.657220 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:06.657314 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:06.657666 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50670: BlockingRecv error: recv got EOF from 127.0.0.1:50670 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 00:59:06.659157 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 32064
I20260810 00:59:06.662775 32515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:06.660153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40824 (local address 127.20.81.129:33307)
0810 00:59:06.660709 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:06.660715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:06.660739 (+ 24us) server_negotiation.cc:407] Connection header received
0810 00:59:06.660792 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:06.660798 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:06.661001 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:06.661117 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:06.662048 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:06.662054 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:06.662065 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:59:06.662079 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:06.662120 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:59:06.662134 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:06.662236 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:06.662322 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 00:59:06.662456 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":272,"threads_started":1}
I20260810 00:59:06.663939 32205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:59:06.746312 32212 heartbeater.cc:503] Master 127.20.81.190:44743 was elected leader, sending a full tablet report...
I20260810 00:59:06.747538 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 32064
W20260810 00:59:06.768034 31981 connection.cc:535] server connection from 127.20.81.129:58867 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 00:59:06.768317 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 32216
I20260810 00:59:06.771981 32517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:06.769289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33050 (local address 127.20.81.130:40777)
0810 00:59:06.769890 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:06.769898 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:06.769925 (+ 27us) server_negotiation.cc:407] Connection header received
0810 00:59:06.769993 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:06.769999 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:06.770263 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:06.770487 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:06.771375 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:06.771382 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:06.771393 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:59:06.771404 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:06.771447 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:59:06.771466 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:06.771572 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:06.771648 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 00:59:06.771733 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":291,"threads_started":1}
I20260810 00:59:06.773198 32351 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:59:06.854486 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 32216
I20260810 00:59:06.876025 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 32360
I20260810 00:59:06.880357 32519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:06.876940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36994 (local address 127.20.81.131:41097)
0810 00:59:06.877674 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:06.877686 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:06.877724 (+ 38us) server_negotiation.cc:407] Connection header received
0810 00:59:06.877810 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:06.877821 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:06.878067 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:06.878261 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:06.879521 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:06.879528 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:06.879540 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:59:06.879553 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:06.879602 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:59:06.879618 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:06.879829 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:06.879909 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 00:59:06.880024 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":308,"threads_started":1}
I20260810 00:59:06.881650 32499 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:59:06.963191 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 32360
I20260810 00:59:06.982256 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 31961
I20260810 00:59:06.983716 32030 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:59:07.087900 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 31961
2026-08-10T00:59:07Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2466 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:59:07 dist-test-slave-gp6n krb5kdc[32525](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:59:07 dist-test-slave-gp6n krb5kdc[32525](info): set up 2 sockets
Aug 10 00:59:07 dist-test-slave-gp6n krb5kdc[32525](info): commencing operation
krb5kdc: starting...
W20260810 00:59:08.289702 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.003s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 00:59:08 dist-test-slave-gp6n krb5kdc[32525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:59:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:59:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 00:59:08.577687 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:42139
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:42139
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:59:08.915527 32544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:59:08.915807 32544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:59:08.915989 32544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:59:08.923079 32544 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:59:08.923179 32544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:59:08.923300 32544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:59:08.923400 32544 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:59:08.923466 32544 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:59:08.933048 32544 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:42139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:42139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:59:08.934437 32544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:59:08.935819 32544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:59:08.947466 32553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:08.947409 32551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:08.947741 32550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:59:08.948253 32544 server_base.cc:1034] running on GCE node
I20260810 00:59:08.948953 32544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:59:08.950335 32544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:59:08.951598 32544 hybrid_clock.cc:648] HybridClock initialized: now 1786323548951557 us; error 35 us; skew 500 ppm
I20260810 00:59:08.951958 32544 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:59:08 dist-test-slave-gp6n krb5kdc[32525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323548, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:59:08.960071 32544 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 00:59:08.961663 32544 webserver.cc:460] Webserver started at http://127.20.81.190:42541/ using document root <none> and password file <none>
I20260810 00:59:08.962442 32544 fs_manager.cc:359] Metadata directory not provided
I20260810 00:59:08.962611 32544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:59:08.962949 32544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:59:08.966053 32544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "06865dbf3e0742df85dbdc6f74afe91e"
format_stamp: "Formatted at 2026-08-10 00:59:08 on dist-test-slave-gp6n"
server_key: "9d5d00a29f3641ba5576bd6d450541c1"
server_key_iv: "34f1fc37546ba7fa23c4d1a8099e1437"
server_key_version: "encryptionkey@0"
I20260810 00:59:08.966889 32544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "06865dbf3e0742df85dbdc6f74afe91e"
format_stamp: "Formatted at 2026-08-10 00:59:08 on dist-test-slave-gp6n"
server_key: "9d5d00a29f3641ba5576bd6d450541c1"
server_key_iv: "34f1fc37546ba7fa23c4d1a8099e1437"
server_key_version: "encryptionkey@0"
I20260810 00:59:08.972317 32544 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 00:59:08.976315 32561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:08.977815 32544 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260810 00:59:08.978117 32544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "06865dbf3e0742df85dbdc6f74afe91e"
format_stamp: "Formatted at 2026-08-10 00:59:08 on dist-test-slave-gp6n"
server_key: "9d5d00a29f3641ba5576bd6d450541c1"
server_key_iv: "34f1fc37546ba7fa23c4d1a8099e1437"
server_key_version: "encryptionkey@0"
I20260810 00:59:08.978367 32544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:59:08.992583 32544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:59:08.996866 32544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:59:08.997162 32544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:59:09.016196 32544 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:42139
I20260810 00:59:09.016237 32616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:42139 every 8 connection(s)
I20260810 00:59:09.017616 32544 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:59:09.021346 32618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:59:09.024878 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 32544
I20260810 00:59:09.025118 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 00:59:09.025985 20806 external_mini_cluster.cc:1428] Setting key b7772a88b51c6b907f5c97476f2f6beb
I20260810 00:59:09.033932 32618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e: Bootstrap starting.
I20260810 00:59:09.037134 32618 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e: Neither blocks nor log segments found. Creating new log.
I20260810 00:59:09.038159 32618 log.cc:824] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e: Log is configured to *not* fsync() on all Append() calls
Aug 10 00:59:09 dist-test-slave-gp6n krb5kdc[32525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:59:09.041380 32618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e: No bootstrap required, opened a new log
I20260810 00:59:09.049526 32622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:09.028123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38642 (local address 127.20.81.190:42139)
0810 00:59:09.028864 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:09.028876 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:09.028918 (+ 42us) server_negotiation.cc:407] Connection header received
0810 00:59:09.030652 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:09.030674 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:09.031070 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:09.031478 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:09.033003 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:09.034324 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:09.035685 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:09.036429 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:09.036638 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:09.040821 (+ 4183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:09.040833 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:09.040852 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:59:09.040903 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:09.044964 (+ 4061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:59:09.045655 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:59:09.045664 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:59:09.045672 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:59:09.045893 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:59:09.046275 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:59:09.046280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:59:09.046283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:59:09.047020 (+ 737us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:09.047343 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:09.047678 (+ 335us) server_negotiation.cc:299] Negotiation successful
0810 00:59:09.048137 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":315,"threads_started":1}
I20260810 00:59:09.052150 32618 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06865dbf3e0742df85dbdc6f74afe91e" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 42139 } }
I20260810 00:59:09.052625 32618 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:59:09.052711 32618 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06865dbf3e0742df85dbdc6f74afe91e, State: Initialized, Role: FOLLOWER
I20260810 00:59:09.053185 32618 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06865dbf3e0742df85dbdc6f74afe91e" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 42139 } }
I20260810 00:59:09.053422 32618 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:59:09.053539 32618 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:59:09.053658 32618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:59:09.056757 32618 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06865dbf3e0742df85dbdc6f74afe91e" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 42139 } }
I20260810 00:59:09.057267 32618 leader_election.cc:302] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06865dbf3e0742df85dbdc6f74afe91e; no voters:
I20260810 00:59:09.057853 32618 leader_election.cc:288] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:59:09.057989 32624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:59:09.059921 32618 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:59:09.062172 32624 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e [term 1 LEADER]: Becoming Leader. State: Replica: 06865dbf3e0742df85dbdc6f74afe91e, State: Running, Role: LEADER
I20260810 00:59:09.062850 32624 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06865dbf3e0742df85dbdc6f74afe91e" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 42139 } }
I20260810 00:59:09.066078 32625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06865dbf3e0742df85dbdc6f74afe91e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06865dbf3e0742df85dbdc6f74afe91e" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 42139 } } }
I20260810 00:59:09.066088 32626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06865dbf3e0742df85dbdc6f74afe91e. Latest consensus state: current_term: 1 leader_uuid: "06865dbf3e0742df85dbdc6f74afe91e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06865dbf3e0742df85dbdc6f74afe91e" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 42139 } } }
I20260810 00:59:09.066586 32625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e [sys.catalog]: This master's current role is: LEADER
I20260810 00:59:09.066624 32626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e [sys.catalog]: This master's current role is: LEADER
W20260810 00:59:09.072965 32646 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 00:59:09.073109 32646 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 00:59:09.073275 32642 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:59:09.074981 32642 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:59:09.083129 32642 catalog_manager.cc:1347] Generated new cluster ID: 1311cdc875b3481082b1fefa61ef702b
I20260810 00:59:09.083272 32642 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:59:09.095263 32642 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:59:09.096742 32642 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:59:09.108287 32642 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e: Generated new TSK 0
I20260810 00:59:09.109421 32642 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.81.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
I20260810 00:59:09.184227 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:42139
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:59:09.532398 32652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:59:09.532675 32652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:59:09.532845 32652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:59:09.540135 32652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:59:09.540239 32652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:59:09.540401 32652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 00:59:09.550379 32652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:42139
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:59:09.551841 32652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:59:09.553164 32652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:59:09.566938 32658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:09.566988 32659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:09.567035 32662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:59:09.567907 32652 server_base.cc:1034] running on GCE node
I20260810 00:59:09.568498 32652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:59:09.569559 32652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:59:09.570761 32652 hybrid_clock.cc:648] HybridClock initialized: now 1786323549570725 us; error 33 us; skew 500 ppm
I20260810 00:59:09.571053 32652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:59:09 dist-test-slave-gp6n krb5kdc[32525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323549, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:59:09.574676 32652 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 00:59:09.576087 32652 webserver.cc:460] Webserver started at http://127.20.81.129:41695/ using document root <none> and password file <none>
I20260810 00:59:09.576779 32652 fs_manager.cc:359] Metadata directory not provided
I20260810 00:59:09.576925 32652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:59:09.577240 32652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:59:09.580375 32652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "4a11667bee31414bb9849b3e00e21648"
format_stamp: "Formatted at 2026-08-10 00:59:09 on dist-test-slave-gp6n"
server_key: "e918985de5c54fb084dbbf0be32e8872"
server_key_iv: "9053ed4cab387329f1b3ad0a640e12f0"
server_key_version: "encryptionkey@0"
I20260810 00:59:09.581153 32652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "4a11667bee31414bb9849b3e00e21648"
format_stamp: "Formatted at 2026-08-10 00:59:09 on dist-test-slave-gp6n"
server_key: "e918985de5c54fb084dbbf0be32e8872"
server_key_iv: "9053ed4cab387329f1b3ad0a640e12f0"
server_key_version: "encryptionkey@0"
I20260810 00:59:09.586776 32652 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 00:59:09.590035 32669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:09.591444 32652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:59:09.591624 32652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "4a11667bee31414bb9849b3e00e21648"
format_stamp: "Formatted at 2026-08-10 00:59:09 on dist-test-slave-gp6n"
server_key: "e918985de5c54fb084dbbf0be32e8872"
server_key_iv: "9053ed4cab387329f1b3ad0a640e12f0"
server_key_version: "encryptionkey@0"
I20260810 00:59:09.591818 32652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:59:09.613543 32652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:59:09.617758 32652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:59:09.618078 32652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:59:09.619138 32652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:59:09.620522 32652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:59:09.620635 32652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:09.620781 32652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:59:09.620854 32652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:09.654780 32652 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:33429
I20260810 00:59:09.654826 329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:33429 every 8 connection(s)
I20260810 00:59:09.656131 32652 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 00:59:09.663152 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 32652
I20260810 00:59:09.663362 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 00:59:09.664101 20806 external_mini_cluster.cc:1428] Setting key c332b277cfef659aaef19521c904a258
Aug 10 00:59:09 dist-test-slave-gp6n krb5kdc[32525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323549, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:59:09.679013 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:09.658869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:36617 (local address 127.20.81.190:42139)
0810 00:59:09.659343 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:09.659349 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:09.660216 (+ 867us) server_negotiation.cc:407] Connection header received
0810 00:59:09.662201 (+ 1985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:09.662207 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:09.662316 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:09.662467 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:09.664218 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:09.665147 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:09.667395 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:09.667802 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:09.667918 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:09.672197 (+ 4279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:09.672206 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:09.672214 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:59:09.672260 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:09.675807 (+ 3547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:59:09.676580 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:59:09.676588 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:59:09.676595 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:59:09.676744 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:59:09.677199 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:59:09.677206 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:59:09.677211 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:59:09.677599 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:09.677752 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:09.678577 (+ 825us) server_negotiation.cc:299] Negotiation successful
0810 00:59:09.678741 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":258,"threads_started":1}
I20260810 00:59:09.680735 335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:09.659278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:42139 (local address 127.20.81.129:36617)
0810 00:59:09.660043 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0810 00:59:09.660099 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 00:59:09.661933 (+ 1834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:59:09.662750 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:59:09.662758 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:59:09.663319 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:59:09.664043 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:09.664064 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:09.665316 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:09.665321 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:09.667236 (+ 1915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:09.667254 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:09.668054 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:09.668063 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:09.668411 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:09.669198 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:59:09.669238 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:59:09.671972 (+ 2734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:59:09.675958 (+ 3986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:59:09.675965 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:59:09.675978 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:59:09.676444 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:59:09.676868 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:59:09.676871 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:59:09.676874 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:59:09.677072 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:59:09.677726 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:59:09.677732 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:59:09.678060 (+ 328us) client_negotiation.cc:769] Sending connection context
0810 00:59:09.678453 (+ 393us) client_negotiation.cc:240] Negotiation successful
0810 00:59:09.679109 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":259,"threads_started":1}
I20260810 00:59:09.682394 330 heartbeater.cc:347] Connected to a master server at 127.20.81.190:42139
I20260810 00:59:09.682737 330 heartbeater.cc:464] Registering TS with master...
I20260810 00:59:09.683542 330 heartbeater.cc:511] Master 127.20.81.190:42139 requested a full tablet report, sending...
I20260810 00:59:09.685639 32579 ts_manager.cc:194] Registered new tserver with Master: 4a11667bee31414bb9849b3e00e21648 (127.20.81.129:33429)
I20260810 00:59:09.687144 32579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:36617
WARNING: no policy specified for kudu/127.20.81.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
I20260810 00:59:09.730266 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:42139
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:59:10.074909 338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:59:10.075237 338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:59:10.075421 338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:59:10.082876 338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:59:10.083017 338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:59:10.083220 338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 00:59:10.092954 338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:42139
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:59:10.094274 338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:59:10.095790 338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:59:10.109283 344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:10.109299 349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:10.109537 345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:59:10.110054 338 server_base.cc:1034] running on GCE node
I20260810 00:59:10.110747 338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:59:10.111641 338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:59:10.112869 338 hybrid_clock.cc:648] HybridClock initialized: now 1786323550112801 us; error 61 us; skew 500 ppm
I20260810 00:59:10.113211 338 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:59:10 dist-test-slave-gp6n krb5kdc[32525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323550, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:59:10.116572 338 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 00:59:10.118094 338 webserver.cc:460] Webserver started at http://127.20.81.130:37221/ using document root <none> and password file <none>
I20260810 00:59:10.118834 338 fs_manager.cc:359] Metadata directory not provided
I20260810 00:59:10.118940 338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:59:10.119201 338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:59:10.122123 338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "6ea5e797b6d9444290b906e4373a4fae"
format_stamp: "Formatted at 2026-08-10 00:59:10 on dist-test-slave-gp6n"
server_key: "983552a901b28e4a7258024e6259876d"
server_key_iv: "67566749e4d7c91549887cca9eb6ef05"
server_key_version: "encryptionkey@0"
I20260810 00:59:10.122963 338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "6ea5e797b6d9444290b906e4373a4fae"
format_stamp: "Formatted at 2026-08-10 00:59:10 on dist-test-slave-gp6n"
server_key: "983552a901b28e4a7258024e6259876d"
server_key_iv: "67566749e4d7c91549887cca9eb6ef05"
server_key_version: "encryptionkey@0"
I20260810 00:59:10.128271 338 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 00:59:10.131999 358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:10.133512 338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260810 00:59:10.133699 338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "6ea5e797b6d9444290b906e4373a4fae"
format_stamp: "Formatted at 2026-08-10 00:59:10 on dist-test-slave-gp6n"
server_key: "983552a901b28e4a7258024e6259876d"
server_key_iv: "67566749e4d7c91549887cca9eb6ef05"
server_key_version: "encryptionkey@0"
I20260810 00:59:10.133911 338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:59:10.146060 338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:59:10.150682 338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:59:10.150998 338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:59:10.151983 338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:59:10.153378 338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:59:10.153484 338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:10.153640 338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:59:10.153714 338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:10.191079 338 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:41249
I20260810 00:59:10.191181 487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:41249 every 8 connection(s)
I20260810 00:59:10.192518 338 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 00:59:10.199859 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 338
I20260810 00:59:10.200069 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 00:59:10.200551 20806 external_mini_cluster.cc:1428] Setting key b21f78832b98a460587228644873ad47
Aug 10 00:59:10 dist-test-slave-gp6n krb5kdc[32525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323550, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:59:10.218048 490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:10.195369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:57181 (local address 127.20.81.190:42139)
0810 00:59:10.195844 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:10.195849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:10.197077 (+ 1228us) server_negotiation.cc:407] Connection header received
0810 00:59:10.199163 (+ 2086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:10.199169 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:10.199270 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:10.199402 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:10.201264 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.202218 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.203998 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.204912 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.205081 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:10.209217 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:10.209227 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:10.209235 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:59:10.209304 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:10.213491 (+ 4187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:59:10.214274 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:59:10.214284 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:59:10.214291 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:59:10.214494 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:59:10.214950 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:59:10.214958 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:59:10.214965 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:59:10.215406 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:10.215582 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:10.216753 (+ 1171us) server_negotiation.cc:299] Negotiation successful
0810 00:59:10.217196 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":242,"threads_started":1}
I20260810 00:59:10.218771 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:10.196002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:42139 (local address 127.20.81.130:57181)
0810 00:59:10.196893 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0810 00:59:10.196949 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 00:59:10.198870 (+ 1921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:59:10.199670 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:59:10.199679 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:59:10.200355 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:59:10.201076 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:10.201096 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.202426 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.202431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:10.203817 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:10.203845 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.205129 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.205134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:10.205332 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:10.206135 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:59:10.206161 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:59:10.208952 (+ 2791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:59:10.213645 (+ 4693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:59:10.213652 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:59:10.213666 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:59:10.214122 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:59:10.214618 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:59:10.214622 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:59:10.214625 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:59:10.214822 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:59:10.215631 (+ 809us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:59:10.215643 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:59:10.216121 (+ 478us) client_negotiation.cc:769] Sending connection context
0810 00:59:10.216468 (+ 347us) client_negotiation.cc:240] Negotiation successful
0810 00:59:10.217105 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":350,"threads_started":1}
I20260810 00:59:10.220872 488 heartbeater.cc:347] Connected to a master server at 127.20.81.190:42139
I20260810 00:59:10.221258 488 heartbeater.cc:464] Registering TS with master...
I20260810 00:59:10.222164 488 heartbeater.cc:511] Master 127.20.81.190:42139 requested a full tablet report, sending...
I20260810 00:59:10.224014 32579 ts_manager.cc:194] Registered new tserver with Master: 6ea5e797b6d9444290b906e4373a4fae (127.20.81.130:41249)
I20260810 00:59:10.225006 32579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:57181
WARNING: no policy specified for kudu/127.20.81.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
I20260810 00:59:10.273679 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:42139
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:59:10.610072 494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:59:10.610423 494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:59:10.610597 494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:59:10.617625 494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:59:10.617776 494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:59:10.617955 494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 00:59:10.627327 494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:42139
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:59:10.628612 494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:59:10.629973 494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:59:10.643251 500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:10.643328 501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:10.643353 505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:59:10.644498 494 server_base.cc:1034] running on GCE node
I20260810 00:59:10.645114 494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:59:10.646088 494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:59:10.647317 494 hybrid_clock.cc:648] HybridClock initialized: now 1786323550647233 us; error 79 us; skew 500 ppm
I20260810 00:59:10.647675 494 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:59:10 dist-test-slave-gp6n krb5kdc[32525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323550, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:59:10.651059 494 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 00:59:10.652632 494 webserver.cc:460] Webserver started at http://127.20.81.131:36395/ using document root <none> and password file <none>
I20260810 00:59:10.653329 494 fs_manager.cc:359] Metadata directory not provided
I20260810 00:59:10.653482 494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:59:10.653784 494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:59:10.656796 494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "b2c3fac4aa5c4298a19d809ce61d497c"
format_stamp: "Formatted at 2026-08-10 00:59:10 on dist-test-slave-gp6n"
server_key: "33b05ee4ae70f8a94fa112cf37f8e760"
server_key_iv: "4a4fabc5bac8d9cb5efe6c3ce102a750"
server_key_version: "encryptionkey@0"
I20260810 00:59:10.657567 494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "b2c3fac4aa5c4298a19d809ce61d497c"
format_stamp: "Formatted at 2026-08-10 00:59:10 on dist-test-slave-gp6n"
server_key: "33b05ee4ae70f8a94fa112cf37f8e760"
server_key_iv: "4a4fabc5bac8d9cb5efe6c3ce102a750"
server_key_version: "encryptionkey@0"
I20260810 00:59:10.662937 494 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260810 00:59:10.666309 514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:10.667650 494 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 00:59:10.667780 494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "b2c3fac4aa5c4298a19d809ce61d497c"
format_stamp: "Formatted at 2026-08-10 00:59:10 on dist-test-slave-gp6n"
server_key: "33b05ee4ae70f8a94fa112cf37f8e760"
server_key_iv: "4a4fabc5bac8d9cb5efe6c3ce102a750"
server_key_version: "encryptionkey@0"
I20260810 00:59:10.667935 494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:59:10.690065 330 heartbeater.cc:503] Master 127.20.81.190:42139 was elected leader, sending a full tablet report...
I20260810 00:59:10.694841 494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:59:10.698989 494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:59:10.699285 494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:59:10.700233 494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:59:10.701596 494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:59:10.701668 494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:10.701838 494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:59:10.701891 494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:10.735538 494 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:42517
I20260810 00:59:10.735597 638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:42517 every 8 connection(s)
I20260810 00:59:10.736867 494 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 00:59:10.742182 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 494
I20260810 00:59:10.742344 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 00:59:10.742830 20806 external_mini_cluster.cc:1428] Setting key 199a74ce845ad283658b38e51dd2cd4a
Aug 10 00:59:10 dist-test-slave-gp6n krb5kdc[32525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323550, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:59:10.758961 641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:10.739695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:39911 (local address 127.20.81.190:42139)
0810 00:59:10.740172 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:10.740177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:10.741170 (+ 993us) server_negotiation.cc:407] Connection header received
0810 00:59:10.743162 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:10.743168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:10.743256 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:10.743399 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:10.745458 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.746388 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.747975 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.748363 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.748494 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:10.753087 (+ 4593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:10.753092 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:10.753096 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:59:10.753128 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:10.755725 (+ 2597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:59:10.756508 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:59:10.756511 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:59:10.756514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:59:10.756634 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:59:10.757118 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:59:10.757124 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:59:10.757130 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:59:10.757452 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:10.757621 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:10.758307 (+ 686us) server_negotiation.cc:299] Negotiation successful
0810 00:59:10.758557 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":234,"threads_started":1}
I20260810 00:59:10.759938 643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:10.740301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:42139 (local address 127.20.81.131:39911)
0810 00:59:10.741001 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0810 00:59:10.741039 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 00:59:10.742852 (+ 1813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:59:10.743673 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:59:10.743688 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:59:10.744468 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:59:10.745282 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:10.745303 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.746578 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.746584 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:10.747820 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:10.747837 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.748602 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.748611 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:10.748843 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:10.749660 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:59:10.749684 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:59:10.752889 (+ 3205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:59:10.755850 (+ 2961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:59:10.755858 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:59:10.755871 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:59:10.756388 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:59:10.756752 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:59:10.756756 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:59:10.756758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:59:10.756985 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:59:10.757587 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:59:10.757595 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:59:10.757942 (+ 347us) client_negotiation.cc:769] Sending connection context
0810 00:59:10.758216 (+ 274us) client_negotiation.cc:240] Negotiation successful
0810 00:59:10.758809 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":270,"threads_started":1}
I20260810 00:59:10.761356 639 heartbeater.cc:347] Connected to a master server at 127.20.81.190:42139
I20260810 00:59:10.761679 639 heartbeater.cc:464] Registering TS with master...
I20260810 00:59:10.762485 639 heartbeater.cc:511] Master 127.20.81.190:42139 requested a full tablet report, sending...
I20260810 00:59:10.763963 32579 ts_manager.cc:194] Registered new tserver with Master: b2c3fac4aa5c4298a19d809ce61d497c (127.20.81.131:42517)
I20260810 00:59:10.764762 32579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:39911
I20260810 00:59:10.770318 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:59:10.787555 641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:10.775054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38646 (local address 127.20.81.190:42139)
0810 00:59:10.775210 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:10.775214 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:10.775793 (+ 579us) server_negotiation.cc:407] Connection header received
0810 00:59:10.775929 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:10.775932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:10.775990 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:10.776068 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:10.777525 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.778274 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.779578 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.779995 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.780098 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:10.782486 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:10.782496 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:10.782501 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:59:10.782545 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:10.785098 (+ 2553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:59:10.785682 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:59:10.785685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:59:10.785688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:59:10.785806 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:59:10.786132 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:59:10.786136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:59:10.786138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:59:10.786545 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:10.786726 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:10.787113 (+ 387us) server_negotiation.cc:299] Negotiation successful
0810 00:59:10.787278 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 00:59:10.791751 32579 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38646:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:59:10.798792 32579 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:59:10.822091 657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:10.811151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42348 (local address 127.20.81.130:41249)
0810 00:59:10.812385 (+ 1234us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:10.812392 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:10.812438 (+ 46us) server_negotiation.cc:407] Connection header received
0810 00:59:10.812538 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:10.812546 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:10.812807 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:10.813039 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:59:10.814852 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.816962 (+ 2110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.819413 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.820856 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.821023 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:10.821232 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:10.821569 (+ 337us) server_negotiation.cc:299] Negotiation successful
0810 00:59:10.821757 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1074,"thread_start_us":933,"threads_started":1}
I20260810 00:59:10.822130 656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:10.811096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:41249 (local address 127.0.0.1:42348)
0810 00:59:10.811878 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0810 00:59:10.811924 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 00:59:10.812115 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:59:10.813397 (+ 1282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:59:10.813408 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:59:10.813463 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:59:10.813941 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:10.814581 (+ 640us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.817152 (+ 2571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.817161 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:10.819257 (+ 2096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:10.819269 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.821104 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.821111 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:10.821310 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:10.821372 (+ 62us) client_negotiation.cc:769] Sending connection context
0810 00:59:10.821500 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 00:59:10.821683 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":3,"thread_start_us":364,"threads_started":1}
I20260810 00:59:10.822947 661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:10.813003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:42517 (local address 127.0.0.1:40326)
0810 00:59:10.813561 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0810 00:59:10.813577 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:59:10.813699 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:59:10.815075 (+ 1376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:59:10.815079 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:59:10.815104 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:59:10.815419 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:10.815427 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.816911 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.816919 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:10.819517 (+ 2598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:10.819531 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.821408 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.821417 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:10.821614 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:10.821647 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 00:59:10.821735 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 00:59:10.822501 (+ 766us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":28,"thread_start_us":797,"threads_started":1}
I20260810 00:59:10.823562 663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:10.813569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40326 (local address 127.20.81.131:42517)
0810 00:59:10.814484 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:10.814492 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:10.814528 (+ 36us) server_negotiation.cc:407] Connection header received
0810 00:59:10.814647 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:10.814654 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:10.814861 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:10.815563 (+ 702us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:59:10.815770 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.816763 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.819684 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.821044 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.821166 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:10.821386 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:10.823015 (+ 1629us) server_negotiation.cc:299] Negotiation successful
0810 00:59:10.823181 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708,"thread_start_us":549,"threads_started":1}
I20260810 00:59:10.825507 659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:10.812362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:33429 (local address 127.0.0.1:41654)
0810 00:59:10.812935 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0810 00:59:10.812952 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:59:10.813056 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:59:10.817968 (+ 4912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:59:10.817982 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:59:10.818016 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:59:10.818548 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:10.818562 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.820341 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.820347 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:10.822091 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:10.822109 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.824770 (+ 2661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.824776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:10.824940 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:10.824971 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 00:59:10.825067 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 00:59:10.825190 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":293,"threads_started":1}
I20260810 00:59:10.825626 660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:10.812624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41654 (local address 127.20.81.129:33429)
0810 00:59:10.817415 (+ 4791us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:10.817423 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:10.817448 (+ 25us) server_negotiation.cc:407] Connection header received
0810 00:59:10.817538 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:10.817548 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:10.817759 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:10.818738 (+ 979us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:59:10.819003 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.820207 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.822303 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.824552 (+ 2249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.824696 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:10.824939 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:10.825139 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 00:59:10.825317 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4616,"thread_start_us":4451,"threads_started":1}
I20260810 00:59:10.826257 414 tablet_service.cc:1467] Processing CreateTablet for tablet cde57b9b0bca4b6f85fdd67f75e97d13 (DEFAULT_TABLE table=test-table [id=6b8b06512f2b460d8bbb7c4c3b6197cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:59:10.827111 566 tablet_service.cc:1467] Processing CreateTablet for tablet cde57b9b0bca4b6f85fdd67f75e97d13 (DEFAULT_TABLE table=test-table [id=6b8b06512f2b460d8bbb7c4c3b6197cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:59:10.827803 414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cde57b9b0bca4b6f85fdd67f75e97d13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:59:10.828377 566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cde57b9b0bca4b6f85fdd67f75e97d13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:59:10.829046 32725 tablet_service.cc:1467] Processing CreateTablet for tablet cde57b9b0bca4b6f85fdd67f75e97d13 (DEFAULT_TABLE table=test-table [id=6b8b06512f2b460d8bbb7c4c3b6197cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:59:10.830338 32725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cde57b9b0bca4b6f85fdd67f75e97d13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:59:10.841116 664 tablet_bootstrap.cc:492] T cde57b9b0bca4b6f85fdd67f75e97d13 P b2c3fac4aa5c4298a19d809ce61d497c: Bootstrap starting.
I20260810 00:59:10.842312 665 tablet_bootstrap.cc:492] T cde57b9b0bca4b6f85fdd67f75e97d13 P 6ea5e797b6d9444290b906e4373a4fae: Bootstrap starting.
I20260810 00:59:10.844934 664 tablet_bootstrap.cc:654] T cde57b9b0bca4b6f85fdd67f75e97d13 P b2c3fac4aa5c4298a19d809ce61d497c: Neither blocks nor log segments found. Creating new log.
I20260810 00:59:10.845800 666 tablet_bootstrap.cc:492] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648: Bootstrap starting.
I20260810 00:59:10.845857 665 tablet_bootstrap.cc:654] T cde57b9b0bca4b6f85fdd67f75e97d13 P 6ea5e797b6d9444290b906e4373a4fae: Neither blocks nor log segments found. Creating new log.
I20260810 00:59:10.846645 664 log.cc:824] T cde57b9b0bca4b6f85fdd67f75e97d13 P b2c3fac4aa5c4298a19d809ce61d497c: Log is configured to *not* fsync() on all Append() calls
I20260810 00:59:10.847154 665 log.cc:824] T cde57b9b0bca4b6f85fdd67f75e97d13 P 6ea5e797b6d9444290b906e4373a4fae: Log is configured to *not* fsync() on all Append() calls
I20260810 00:59:10.849303 666 tablet_bootstrap.cc:654] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648: Neither blocks nor log segments found. Creating new log.
I20260810 00:59:10.850056 664 tablet_bootstrap.cc:492] T cde57b9b0bca4b6f85fdd67f75e97d13 P b2c3fac4aa5c4298a19d809ce61d497c: No bootstrap required, opened a new log
I20260810 00:59:10.850329 664 ts_tablet_manager.cc:1397] T cde57b9b0bca4b6f85fdd67f75e97d13 P b2c3fac4aa5c4298a19d809ce61d497c: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260810 00:59:10.850382 666 log.cc:824] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648: Log is configured to *not* fsync() on all Append() calls
I20260810 00:59:10.850481 665 tablet_bootstrap.cc:492] T cde57b9b0bca4b6f85fdd67f75e97d13 P 6ea5e797b6d9444290b906e4373a4fae: No bootstrap required, opened a new log
I20260810 00:59:10.850744 665 ts_tablet_manager.cc:1397] T cde57b9b0bca4b6f85fdd67f75e97d13 P 6ea5e797b6d9444290b906e4373a4fae: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.005s
I20260810 00:59:10.852999 666 tablet_bootstrap.cc:492] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648: No bootstrap required, opened a new log
I20260810 00:59:10.853236 666 ts_tablet_manager.cc:1397] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260810 00:59:10.860270 666 raft_consensus.cc:348] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ea5e797b6d9444290b906e4373a4fae" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41249 } } peers { permanent_uuid: "4a11667bee31414bb9849b3e00e21648" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33429 } } peers { permanent_uuid: "b2c3fac4aa5c4298a19d809ce61d497c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42517 } }
I20260810 00:59:10.860724 666 raft_consensus.cc:374] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:59:10.860826 666 raft_consensus.cc:729] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a11667bee31414bb9849b3e00e21648, State: Initialized, Role: FOLLOWER
I20260810 00:59:10.861202 664 raft_consensus.cc:348] T cde57b9b0bca4b6f85fdd67f75e97d13 P b2c3fac4aa5c4298a19d809ce61d497c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ea5e797b6d9444290b906e4373a4fae" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41249 } } peers { permanent_uuid: "4a11667bee31414bb9849b3e00e21648" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33429 } } peers { permanent_uuid: "b2c3fac4aa5c4298a19d809ce61d497c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42517 } }
I20260810 00:59:10.861487 666 consensus_queue.cc:260] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ea5e797b6d9444290b906e4373a4fae" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41249 } } peers { permanent_uuid: "4a11667bee31414bb9849b3e00e21648" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33429 } } peers { permanent_uuid: "b2c3fac4aa5c4298a19d809ce61d497c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42517 } }
I20260810 00:59:10.861384 665 raft_consensus.cc:348] T cde57b9b0bca4b6f85fdd67f75e97d13 P 6ea5e797b6d9444290b906e4373a4fae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ea5e797b6d9444290b906e4373a4fae" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41249 } } peers { permanent_uuid: "4a11667bee31414bb9849b3e00e21648" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33429 } } peers { permanent_uuid: "b2c3fac4aa5c4298a19d809ce61d497c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42517 } }
I20260810 00:59:10.861668 664 raft_consensus.cc:374] T cde57b9b0bca4b6f85fdd67f75e97d13 P b2c3fac4aa5c4298a19d809ce61d497c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:59:10.861766 664 raft_consensus.cc:729] T cde57b9b0bca4b6f85fdd67f75e97d13 P b2c3fac4aa5c4298a19d809ce61d497c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c3fac4aa5c4298a19d809ce61d497c, State: Initialized, Role: FOLLOWER
I20260810 00:59:10.861792 665 raft_consensus.cc:374] T cde57b9b0bca4b6f85fdd67f75e97d13 P 6ea5e797b6d9444290b906e4373a4fae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:59:10.861889 665 raft_consensus.cc:729] T cde57b9b0bca4b6f85fdd67f75e97d13 P 6ea5e797b6d9444290b906e4373a4fae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ea5e797b6d9444290b906e4373a4fae, State: Initialized, Role: FOLLOWER
I20260810 00:59:10.862466 665 consensus_queue.cc:260] T cde57b9b0bca4b6f85fdd67f75e97d13 P 6ea5e797b6d9444290b906e4373a4fae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ea5e797b6d9444290b906e4373a4fae" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41249 } } peers { permanent_uuid: "4a11667bee31414bb9849b3e00e21648" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33429 } } peers { permanent_uuid: "b2c3fac4aa5c4298a19d809ce61d497c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42517 } }
I20260810 00:59:10.862552 664 consensus_queue.cc:260] T cde57b9b0bca4b6f85fdd67f75e97d13 P b2c3fac4aa5c4298a19d809ce61d497c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ea5e797b6d9444290b906e4373a4fae" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41249 } } peers { permanent_uuid: "4a11667bee31414bb9849b3e00e21648" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33429 } } peers { permanent_uuid: "b2c3fac4aa5c4298a19d809ce61d497c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42517 } }
I20260810 00:59:10.863973 639 heartbeater.cc:503] Master 127.20.81.190:42139 was elected leader, sending a full tablet report...
I20260810 00:59:10.864019 488 heartbeater.cc:503] Master 127.20.81.190:42139 was elected leader, sending a full tablet report...
I20260810 00:59:10.864434 664 ts_tablet_manager.cc:1428] T cde57b9b0bca4b6f85fdd67f75e97d13 P b2c3fac4aa5c4298a19d809ce61d497c: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260810 00:59:10.865378 665 ts_tablet_manager.cc:1428] T cde57b9b0bca4b6f85fdd67f75e97d13 P 6ea5e797b6d9444290b906e4373a4fae: Time spent starting tablet: real 0.015s user 0.011s sys 0.002s
I20260810 00:59:10.865967 666 ts_tablet_manager.cc:1428] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
W20260810 00:59:10.908656 332 tablet.cc:2369] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:59:10.927349 670 raft_consensus.cc:482] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:59:10.927685 670 raft_consensus.cc:504] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ea5e797b6d9444290b906e4373a4fae" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41249 } } peers { permanent_uuid: "4a11667bee31414bb9849b3e00e21648" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33429 } } peers { permanent_uuid: "b2c3fac4aa5c4298a19d809ce61d497c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42517 } }
I20260810 00:59:10.929347 670 leader_election.cc:288] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ea5e797b6d9444290b906e4373a4fae (127.20.81.130:41249), b2c3fac4aa5c4298a19d809ce61d497c (127.20.81.131:42517)
I20260810 00:59:10.936452 657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:10.929849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:45795 (local address 127.20.81.130:41249)
0810 00:59:10.930143 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:10.930151 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:10.930570 (+ 419us) server_negotiation.cc:407] Connection header received
0810 00:59:10.930815 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:10.930823 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:10.930917 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:10.931015 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:59:10.931842 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.932816 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.934677 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.935458 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.935639 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:10.935722 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:10.936041 (+ 319us) server_negotiation.cc:299] Negotiation successful
0810 00:59:10.936168 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260810 00:59:10.936581 673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:10.929647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:41249 (local address 127.20.81.129:45795)
0810 00:59:10.930460 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0810 00:59:10.930481 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:59:10.930653 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:59:10.931148 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:59:10.931153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:59:10.931183 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:59:10.931661 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:10.931676 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.932952 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.932959 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:10.934518 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:10.934533 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.935683 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.935689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:10.935872 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:10.935896 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 00:59:10.936015 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 00:59:10.936151 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"mutex_wait_us":22,"thread_start_us":399,"threads_started":1}
I20260810 00:59:10.936671 663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:10.930801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:44843 (local address 127.20.81.131:42517)
0810 00:59:10.931025 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:10.931031 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:10.931392 (+ 361us) server_negotiation.cc:407] Connection header received
0810 00:59:10.931603 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:10.931607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:10.931667 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:10.931762 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:59:10.932271 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.933081 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.934912 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.935717 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.935893 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:10.935967 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:10.936289 (+ 322us) server_negotiation.cc:299] Negotiation successful
0810 00:59:10.936419 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 00:59:10.937076 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:10.930480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:42517 (local address 127.20.81.129:44843)
0810 00:59:10.931277 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0810 00:59:10.931290 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 00:59:10.931447 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:59:10.931777 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:59:10.931781 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:59:10.931803 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:59:10.932125 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:10.932133 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.933221 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.933227 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:10.934759 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:59:10.934772 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.935936 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.935942 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:59:10.936136 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:10.936160 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 00:59:10.936263 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 00:59:10.936390 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"mutex_wait_us":46,"thread_start_us":424,"threads_started":1}
I20260810 00:59:10.937642 437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde57b9b0bca4b6f85fdd67f75e97d13" candidate_uuid: "4a11667bee31414bb9849b3e00e21648" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ea5e797b6d9444290b906e4373a4fae" is_pre_election: true
I20260810 00:59:10.938076 586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde57b9b0bca4b6f85fdd67f75e97d13" candidate_uuid: "4a11667bee31414bb9849b3e00e21648" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c3fac4aa5c4298a19d809ce61d497c" is_pre_election: true
I20260810 00:59:10.938192 437 raft_consensus.cc:2454] T cde57b9b0bca4b6f85fdd67f75e97d13 P 6ea5e797b6d9444290b906e4373a4fae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a11667bee31414bb9849b3e00e21648 in term 0.
I20260810 00:59:10.938622 586 raft_consensus.cc:2454] T cde57b9b0bca4b6f85fdd67f75e97d13 P b2c3fac4aa5c4298a19d809ce61d497c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a11667bee31414bb9849b3e00e21648 in term 0.
I20260810 00:59:10.938951 32673 leader_election.cc:302] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a11667bee31414bb9849b3e00e21648, 6ea5e797b6d9444290b906e4373a4fae; no voters:
I20260810 00:59:10.939345 670 raft_consensus.cc:2781] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:59:10.939519 670 raft_consensus.cc:482] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:59:10.939641 670 raft_consensus.cc:3037] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:59:10.941861 670 raft_consensus.cc:504] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ea5e797b6d9444290b906e4373a4fae" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41249 } } peers { permanent_uuid: "4a11667bee31414bb9849b3e00e21648" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33429 } } peers { permanent_uuid: "b2c3fac4aa5c4298a19d809ce61d497c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42517 } }
I20260810 00:59:10.942507 670 leader_election.cc:288] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648 [CANDIDATE]: Term 1 election: Requested vote from peers 6ea5e797b6d9444290b906e4373a4fae (127.20.81.130:41249), b2c3fac4aa5c4298a19d809ce61d497c (127.20.81.131:42517)
I20260810 00:59:10.942943 437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde57b9b0bca4b6f85fdd67f75e97d13" candidate_uuid: "4a11667bee31414bb9849b3e00e21648" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ea5e797b6d9444290b906e4373a4fae"
I20260810 00:59:10.943140 437 raft_consensus.cc:3037] T cde57b9b0bca4b6f85fdd67f75e97d13 P 6ea5e797b6d9444290b906e4373a4fae [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:59:10.943171 586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde57b9b0bca4b6f85fdd67f75e97d13" candidate_uuid: "4a11667bee31414bb9849b3e00e21648" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c3fac4aa5c4298a19d809ce61d497c"
I20260810 00:59:10.943390 586 raft_consensus.cc:3037] T cde57b9b0bca4b6f85fdd67f75e97d13 P b2c3fac4aa5c4298a19d809ce61d497c [term 0 FOLLOWER]: Advancing to term 1
W20260810 00:59:10.944638 489 tablet.cc:2369] T cde57b9b0bca4b6f85fdd67f75e97d13 P 6ea5e797b6d9444290b906e4373a4fae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:59:10.945640 586 raft_consensus.cc:2454] T cde57b9b0bca4b6f85fdd67f75e97d13 P b2c3fac4aa5c4298a19d809ce61d497c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a11667bee31414bb9849b3e00e21648 in term 1.
I20260810 00:59:10.946048 32673 leader_election.cc:302] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a11667bee31414bb9849b3e00e21648, b2c3fac4aa5c4298a19d809ce61d497c; no voters:
I20260810 00:59:10.946246 437 raft_consensus.cc:2454] T cde57b9b0bca4b6f85fdd67f75e97d13 P 6ea5e797b6d9444290b906e4373a4fae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a11667bee31414bb9849b3e00e21648 in term 1.
I20260810 00:59:10.946324 670 raft_consensus.cc:2781] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:59:10.946885 670 raft_consensus.cc:686] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648 [term 1 LEADER]: Becoming Leader. State: Replica: 4a11667bee31414bb9849b3e00e21648, State: Running, Role: LEADER
I20260810 00:59:10.947427 670 consensus_queue.cc:237] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ea5e797b6d9444290b906e4373a4fae" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41249 } } peers { permanent_uuid: "4a11667bee31414bb9849b3e00e21648" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33429 } } peers { permanent_uuid: "b2c3fac4aa5c4298a19d809ce61d497c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42517 } }
I20260810 00:59:10.951901 32579 catalog_manager.cc:5495] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a11667bee31414bb9849b3e00e21648 (127.20.81.129). New cstate: current_term: 1 leader_uuid: "4a11667bee31414bb9849b3e00e21648" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ea5e797b6d9444290b906e4373a4fae" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a11667bee31414bb9849b3e00e21648" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2c3fac4aa5c4298a19d809ce61d497c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42517 } health_report { overall_health: UNKNOWN } } }
I20260810 00:59:10.974887 660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:10.969322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41670 (local address 127.20.81.129:33429)
0810 00:59:10.969541 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:10.969547 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:10.969569 (+ 22us) server_negotiation.cc:407] Connection header received
0810 00:59:10.969680 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:10.969683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:10.969778 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:10.969963 (+ 185us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:59:10.970361 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.971203 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.972627 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:10.973127 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:10.973258 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:10.973753 (+ 495us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:59:10.974266 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:59:10.974467 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:10.974529 (+ 62us) server_negotiation.cc:299] Negotiation successful
0810 00:59:10.974612 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 00:59:10.983592 586 raft_consensus.cc:1261] T cde57b9b0bca4b6f85fdd67f75e97d13 P b2c3fac4aa5c4298a19d809ce61d497c [term 1 FOLLOWER]: Refusing update from remote peer 4a11667bee31414bb9849b3e00e21648: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:59:10.983592 437 raft_consensus.cc:1261] T cde57b9b0bca4b6f85fdd67f75e97d13 P 6ea5e797b6d9444290b906e4373a4fae [term 1 FOLLOWER]: Refusing update from remote peer 4a11667bee31414bb9849b3e00e21648: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:59:10.984750 675 consensus_queue.cc:1035] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2c3fac4aa5c4298a19d809ce61d497c" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:59:10.985185 670 consensus_queue.cc:1035] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ea5e797b6d9444290b906e4373a4fae" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 00:59:10.988556 640 tablet.cc:2369] T cde57b9b0bca4b6f85fdd67f75e97d13 P b2c3fac4aa5c4298a19d809ce61d497c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:59:11.000388 679 mvcc.cc:205] Tried to move back new op lower bound from 7316781264817680384 to 7316781264684130304. Current Snapshot: MvccSnapshot[applied={T|T < 7316781264817680384}]
I20260810 00:59:11.014348 32578 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:59:11.014719 32578 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:59:11.019209 32578 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 06865dbf3e0742df85dbdc6f74afe91e: Sending DeleteTablet for 3 replicas of tablet cde57b9b0bca4b6f85fdd67f75e97d13
I20260810 00:59:11.020584 414 tablet_service.cc:1514] Processing DeleteTablet for tablet cde57b9b0bca4b6f85fdd67f75e97d13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:59:11 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:42348
I20260810 00:59:11.020691 566 tablet_service.cc:1514] Processing DeleteTablet for tablet cde57b9b0bca4b6f85fdd67f75e97d13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:59:11 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:40326
I20260810 00:59:11.021420 32725 tablet_service.cc:1514] Processing DeleteTablet for tablet cde57b9b0bca4b6f85fdd67f75e97d13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:59:11 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:41654
I20260810 00:59:11.021545 690 tablet_replica.cc:330] T cde57b9b0bca4b6f85fdd67f75e97d13 P b2c3fac4aa5c4298a19d809ce61d497c: stopping tablet replica
I20260810 00:59:11.022056 690 raft_consensus.cc:2229] T cde57b9b0bca4b6f85fdd67f75e97d13 P b2c3fac4aa5c4298a19d809ce61d497c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:59:11.022668 690 raft_consensus.cc:2258] T cde57b9b0bca4b6f85fdd67f75e97d13 P b2c3fac4aa5c4298a19d809ce61d497c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:59:11.023485 689 tablet_replica.cc:330] T cde57b9b0bca4b6f85fdd67f75e97d13 P 6ea5e797b6d9444290b906e4373a4fae: stopping tablet replica
I20260810 00:59:11.023677 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 32652
I20260810 00:59:11.024055 689 raft_consensus.cc:2229] T cde57b9b0bca4b6f85fdd67f75e97d13 P 6ea5e797b6d9444290b906e4373a4fae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:59:11.024595 689 raft_consensus.cc:2258] T cde57b9b0bca4b6f85fdd67f75e97d13 P 6ea5e797b6d9444290b906e4373a4fae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:59:11.026840 689 ts_tablet_manager.cc:1905] T cde57b9b0bca4b6f85fdd67f75e97d13 P 6ea5e797b6d9444290b906e4373a4fae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:59:11.028215 690 ts_tablet_manager.cc:1905] T cde57b9b0bca4b6f85fdd67f75e97d13 P b2c3fac4aa5c4298a19d809ce61d497c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:59:11.035413 690 ts_tablet_manager.cc:1918] T cde57b9b0bca4b6f85fdd67f75e97d13 P b2c3fac4aa5c4298a19d809ce61d497c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:59:11.043236 689 ts_tablet_manager.cc:1918] T cde57b9b0bca4b6f85fdd67f75e97d13 P 6ea5e797b6d9444290b906e4373a4fae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:59:11.043541 690 log.cc:1196] T cde57b9b0bca4b6f85fdd67f75e97d13 P b2c3fac4aa5c4298a19d809ce61d497c: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/cde57b9b0bca4b6f85fdd67f75e97d13
I20260810 00:59:11.044193 690 ts_tablet_manager.cc:1939] T cde57b9b0bca4b6f85fdd67f75e97d13 P b2c3fac4aa5c4298a19d809ce61d497c: Deleting consensus metadata
I20260810 00:59:11.046171 689 log.cc:1196] T cde57b9b0bca4b6f85fdd67f75e97d13 P 6ea5e797b6d9444290b906e4373a4fae: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/cde57b9b0bca4b6f85fdd67f75e97d13
I20260810 00:59:11.046720 32565 catalog_manager.cc:4841] TS b2c3fac4aa5c4298a19d809ce61d497c (127.20.81.131:42517): tablet cde57b9b0bca4b6f85fdd67f75e97d13 (table test-table [id=6b8b06512f2b460d8bbb7c4c3b6197cc]) successfully deleted
I20260810 00:59:11.046835 689 ts_tablet_manager.cc:1939] T cde57b9b0bca4b6f85fdd67f75e97d13 P 6ea5e797b6d9444290b906e4373a4fae: Deleting consensus metadata
I20260810 00:59:11.048682 32565 catalog_manager.cc:4841] TS 6ea5e797b6d9444290b906e4373a4fae (127.20.81.130:41249): tablet cde57b9b0bca4b6f85fdd67f75e97d13 (table test-table [id=6b8b06512f2b460d8bbb7c4c3b6197cc]) successfully deleted
I20260810 00:59:11.049664 691 tablet_replica.cc:330] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648: stopping tablet replica
I20260810 00:59:11.050194 691 raft_consensus.cc:2229] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:59:11.051066 691 raft_consensus.cc:2258] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:59:11.053139 691 ts_tablet_manager.cc:1905] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 00:59:11 dist-test-slave-gp6n krb5kdc[32525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.129@KRBTEST.COM
I20260810 00:59:11.060469 691 ts_tablet_manager.cc:1918] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:59:11.060672 691 log.cc:1196] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323445691443-20806-0/minicluster-data/ts-0/wal/wals/cde57b9b0bca4b6f85fdd67f75e97d13
I20260810 00:59:11.061367 691 ts_tablet_manager.cc:1939] T cde57b9b0bca4b6f85fdd67f75e97d13 P 4a11667bee31414bb9849b3e00e21648: Deleting consensus metadata
I20260810 00:59:11.063421 32565 catalog_manager.cc:4841] TS 4a11667bee31414bb9849b3e00e21648 (127.20.81.129:33429): tablet cde57b9b0bca4b6f85fdd67f75e97d13 (table test-table [id=6b8b06512f2b460d8bbb7c4c3b6197cc]) successfully deleted
I20260810 00:59:11.067670 660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:11.025102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41686 (local address 127.20.81.129:33429)
0810 00:59:11.025274 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:11.025280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:11.048842 (+ 23562us) server_negotiation.cc:407] Connection header received
0810 00:59:11.049110 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:11.049120 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:11.049231 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:11.049383 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:11.051180 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:11.052404 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:11.054257 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:11.055053 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:11.055191 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:11.060756 (+ 5565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:11.060773 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:11.060821 (+ 48us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:59:11.060887 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:11.064822 (+ 3935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:59:11.065666 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:59:11.065672 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:59:11.065675 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:59:11.065811 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:59:11.066207 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:59:11.066210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:59:11.066212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:59:11.066639 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:11.066884 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:11.067207 (+ 323us) server_negotiation.cc:299] Negotiation successful
0810 00:59:11.067389 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 00:59:11.068320 324 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:59:11.169044 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 32652
I20260810 00:59:11.190783 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 338
Aug 10 00:59:11 dist-test-slave-gp6n krb5kdc[32525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.130@KRBTEST.COM
I20260810 00:59:11.206251 657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:11.191627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42360 (local address 127.20.81.130:41249)
0810 00:59:11.191800 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:11.191808 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:11.191867 (+ 59us) server_negotiation.cc:407] Connection header received
0810 00:59:11.192089 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:11.192099 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:11.192162 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:11.192267 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:11.193510 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:11.194307 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:11.195816 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:11.196222 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:11.196358 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:11.200414 (+ 4056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:11.200426 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:11.200439 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:59:11.200473 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:11.203532 (+ 3059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:59:11.204176 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:59:11.204182 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:59:11.204185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:59:11.204300 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:59:11.204688 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:59:11.204693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:59:11.204698 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:59:11.205094 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:11.205383 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:11.205703 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 00:59:11.205950 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 00:59:11.206899 483 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:59:11.297147 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 338
I20260810 00:59:11.318687 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 494
Aug 10 00:59:11 dist-test-slave-gp6n krb5kdc[32525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.131@KRBTEST.COM
I20260810 00:59:11.334381 663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:11.319547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40330 (local address 127.20.81.131:42517)
0810 00:59:11.319748 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:11.319754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:11.319819 (+ 65us) server_negotiation.cc:407] Connection header received
0810 00:59:11.320048 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:11.320053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:11.320119 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:11.320228 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:11.321584 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:11.322328 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:11.323601 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:11.324025 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:11.324162 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:11.328770 (+ 4608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:11.328785 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:11.328801 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:59:11.328840 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:11.331663 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:59:11.332331 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:59:11.332339 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:59:11.332341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:59:11.332488 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:59:11.332853 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:59:11.332859 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:59:11.332862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:59:11.333222 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:11.333535 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:11.333827 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 00:59:11.334064 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 00:59:11.335103 633 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:59:11.424176 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 494
I20260810 00:59:11.444079 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 32544
I20260810 00:59:11.445355 32612 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:59:11.557035 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 32544
2026-08-10T00:59:11Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4468 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:59:11 dist-test-slave-gp6n krb5kdc[699](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:59:11 dist-test-slave-gp6n krb5kdc[699](info): set up 2 sockets
Aug 10 00:59:11 dist-test-slave-gp6n krb5kdc[699](info): commencing operation
krb5kdc: starting...
W20260810 00:59:12.766350 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 00:59:12 dist-test-slave-gp6n krb5kdc[699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:59:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:59:12Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.20.81.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.20.81.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.20.81.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/rangeradmin_127.20.81.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.20.81.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/rangeradmin_127.20.81.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.20.81.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.20.81.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.20.81.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/rangerlookup_127.20.81.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.20.81.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/rangerlookup_127.20.81.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.81.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.81.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.81.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/HTTP_127.20.81.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.81.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/HTTP_127.20.81.148@KRBTEST.COM.keytab.
I20260810 00:59:13.169812 20806 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task5NFMaT/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/postgres -l logfile start
2026-08-10 00:59:15.869 UTC [732] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 00:59:15.869 UTC [732] LOG: listening on IPv4 address "127.20.81.148", port 34677
2026-08-10 00:59:15.873 UTC [732] LOG: listening on Unix socket "/tmp/.s.PGSQL.34677"
2026-08-10 00:59:15.899 UTC [735] LOG: database system was shut down at 2026-08-10 00:59:14 UTC
2026-08-10 00:59:15.903 UTC [732] LOG: database system is ready to accept connections
I20260810 00:59:16.947940 20806 mini_postgres.cc:96] Postgres bound to 34677
2026-08-10 00:59:16.972 UTC [743] FATAL: database "slave" does not exist
127.20.81.148:34677 - accepting connections
I20260810 00:59:16.973948 20806 mini_ranger.cc:161] Starting Ranger...
I20260810 00:59:17.008034 20806 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 00:59:17.536950 20806 mini_ranger.cc:87] Created ranger Postgres database
I20260810 00:59:17.537117 20806 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ranger-admin
2026-08-10 00:59:17,856 [I] DB FLAVOR :POSTGRES
2026-08-10 00:59:17,856 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 00:59:17,857 [I] Checking connection..
2026-08-10 00:59:17,857 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 00:59:18,098 [I] Checking connection passed.
2026-08-10 00:59:18,098 [I] --------- Verifying version history table ---------
2026-08-10 00:59:18,098 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 00:59:18,343 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 00:59:18,343 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 00:59:18,609 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 00:59:18,609 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 00:59:18,609 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 00:59:18.860 UTC [819] WARNING: there is no transaction in progress
2026-08-10 00:59:18,866 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 00:59:18,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 00:59:19,124 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 00:59:19,124 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 00:59:19,124 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 00:59:19,376 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 00:59:19,643 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gp6n.c.gcp-upstream.internal','N') ;"
2026-08-10 00:59:19,877 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 00:59:20,125 [I] Table x_portal_user does not exist in database ranger
2026-08-10 00:59:20,126 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 00:59:20,373 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 00:59:20,373 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 00:59:20,628 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 00:59:20,868 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 00:59:20,868 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 00:59:21.866 UTC [939] WARNING: there is no transaction in progress
2026-08-10 00:59:21.883 UTC [939] WARNING: there is no transaction in progress
2026-08-10 00:59:21.900 UTC [939] WARNING: there is no transaction in progress
2026-08-10 00:59:21.916 UTC [939] WARNING: there is no transaction in progress
2026-08-10 00:59:21.934 UTC [939] WARNING: there is no transaction in progress
2026-08-10 00:59:21.953 UTC [939] WARNING: there is no transaction in progress
2026-08-10 00:59:22.010 UTC [939] WARNING: there is no transaction in progress
2026-08-10 00:59:22.017 UTC [939] WARNING: there is no transaction in progress
2026-08-10 00:59:22.033 UTC [939] WARNING: there is no transaction in progress
2026-08-10 00:59:22.044 UTC [939] WARNING: there is no transaction in progress
2026-08-10 00:59:22.056 UTC [939] WARNING: there is no transaction in progress
2026-08-10 00:59:22.070 UTC [939] WARNING: there is no transaction in progress
2026-08-10 00:59:22.080 UTC [939] WARNING: there is no transaction in progress
2026-08-10 00:59:22.086 UTC [939] WARNING: there is no transaction in progress
2026-08-10 00:59:22.093 UTC [939] WARNING: there is no transaction in progress
2026-08-10 00:59:22.100 UTC [939] WARNING: there is no transaction in progress
2026-08-10 00:59:22,789 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 00:59:22,789 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 00:59:23,068 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 00:59:23,069 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 00:59:23,342 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 00:59:23,342 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 00:59:23,590 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 00:59:23,590 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 00:59:23,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 00:59:24,087 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gp6n.c.gcp-upstream.internal';"
2026-08-10 00:59:24,350 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 00:59:24,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 00:59:24,592 [I] DB_PATCHES have already been applied
I20260810 00:59:24.598496 20806 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ranger-admin:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task5NFMaT/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 00:59:24.598610 20806 mini_ranger.cc:193] Using host: 127.20.81.148
I20260810 00:59:24.620811 20806 mini_ranger.cc:227] Ranger admin URL: http://127.20.81.148:35939
Aug 10, 2026 12:59:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 12:59:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 12:59:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 12:59:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 12:59:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 12:59:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 12:59:25 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 00:59:25.893253 20806 mini_ranger.cc:160] Time spent starting Ranger: real 8.919s user 0.009s sys 0.057s
Aug 10, 2026 12:59:45 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 12:59:45 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Aug 10, 2026 12:59:45 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 12:59:46 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 12:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 12:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 12:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 12:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 12:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 12:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 12:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 12:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 12:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 12:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 12:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 12:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 12:59:46 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 12:59:46 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 00:59:47.585062 20806 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.20.81.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.20.81.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.20.81.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/rangerkms_127.20.81.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.20.81.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/rangerkms_127.20.81.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.81.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.81.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.81.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/HTTP_127.20.81.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.81.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/HTTP_127.20.81.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 00:59:47 dist-test-slave-gp6n krb5kdc[699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323587, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 00:59:47.807544 20806 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 00:59:47.842788 20806 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 00:59:48.795114 20806 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 00:59:48.795248 20806 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ranger-kms
I20260810 00:59:48.795353 20806 mini_ranger_kms.cc:227] Using postgres at 127.20.81.148:34677
2026-08-10 00:59:49,288 [I] DB FLAVOR :POSTGRES
2026-08-10 00:59:49,288 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 00:59:49,289 [I] Checking connection
2026-08-10 00:59:49,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 00:59:49,525 [I] connection success
2026-08-10 00:59:49,526 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 00:59:49,526 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 00:59:49,526 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 00:59:49,768 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 00:59:49,768 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 00:59:49,768 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 00:59:49,769 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:34677/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 00:59:50,035 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 00:59:50.513516 20806 mini_ranger_kms.cc:310] Created kms service
I20260810 00:59:50.753309 20806 mini_ranger_kms.cc:326] Created kudu user
I20260810 00:59:50.801622 20806 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 00:59:50 dist-test-slave-gp6n krb5kdc[699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323587, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
I20260810 00:59:51.204609 20806 mini_ranger_kms.cc:384] Added ranger policy
I20260810 00:59:51.205045 20806 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ranger-kms:/tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 00:59:51.205133 20806 mini_ranger_kms.cc:240] Using host: 127.20.81.148
I20260810 00:59:51.236176 20806 mini_ranger_kms.cc:276] Ranger KMS PID: 1144
I20260810 00:59:51.236418 20806 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.20.81.148:58115
Aug 10, 2026 12:59:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 12:59:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 12:59:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 12:59:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 12:59:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 12:59:52 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 00:59:53 dist-test-slave-gp6n krb5kdc[699](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786323593, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.81.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:59:53.751598 20806 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.944s user 0.129s sys 0.404s
I20260810 00:59:53.751916 20806 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 00:59:53.752169 20806 mini_ranger_kms.cc:401] 127.20.81.148:58115/kms/v1/keys
Aug 10 00:59:54 dist-test-slave-gp6n krb5kdc[699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323593, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.81.148@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
Aug 10 00:59:54 dist-test-slave-gp6n krb5kdc[699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323593, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.81.148@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
Aug 10 00:59:56 dist-test-slave-gp6n krb5kdc[699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 00:59:56.856865 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:43837
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:41343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:43837
--ranger_config_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.81.148:58115/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:59:57.459502 1200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:59:57.459933 1200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:59:57.460160 1200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:59:57.469841 1200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:59:57.469998 1200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:59:57.470131 1200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:59:57.470255 1200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:59:57.470376 1200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:59:57.483095 1200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:41343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/master-0/wal
--ranger_kms_url=127.20.81.148:58115/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:43837
--ranger_config_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:43837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:59:57.484674 1200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:59:57.486160 1200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:59:57.505148 1205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:57.505779 1206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:57.506080 1208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:59:57.507869 1200 server_base.cc:1034] running on GCE node
I20260810 00:59:57.508967 1200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:59:57.511150 1200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:59:57.512495 1200 hybrid_clock.cc:648] HybridClock initialized: now 1786323597512334 us; error 92 us; skew 500 ppm
I20260810 00:59:57.513003 1200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:59:57 dist-test-slave-gp6n krb5kdc[699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323597, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:59:57.517827 1200 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 00:59:57.521401 1200 webserver.cc:460] Webserver started at http://127.20.81.190:40283/ using document root <none> and password file <none>
I20260810 00:59:57.522262 1200 fs_manager.cc:359] Metadata directory not provided
I20260810 00:59:57.522490 1200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:59:57.522897 1200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 00:59:57 dist-test-slave-gp6n krb5kdc[699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323597, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
I20260810 00:59:57.586258 1200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "1c7d268f14f84181ab248f3021cb52b1"
format_stamp: "Formatted at 2026-08-10 00:59:57 on dist-test-slave-gp6n"
server_key: "d070874b75b6e6f90591be12ad9d3614"
server_key_iv: "36ff2616d481b8ec58a24f0480a9d8b8"
server_key_version: "kuduclusterkey@0"
I20260810 00:59:57.587298 1200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "1c7d268f14f84181ab248f3021cb52b1"
format_stamp: "Formatted at 2026-08-10 00:59:57 on dist-test-slave-gp6n"
server_key: "d070874b75b6e6f90591be12ad9d3614"
server_key_iv: "36ff2616d481b8ec58a24f0480a9d8b8"
server_key_version: "kuduclusterkey@0"
I20260810 00:59:57.594960 1200 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260810 00:59:57.614761 1215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:57.616520 1200 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260810 00:59:57.616745 1200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "1c7d268f14f84181ab248f3021cb52b1"
format_stamp: "Formatted at 2026-08-10 00:59:57 on dist-test-slave-gp6n"
server_key: "d070874b75b6e6f90591be12ad9d3614"
server_key_iv: "36ff2616d481b8ec58a24f0480a9d8b8"
server_key_version: "kuduclusterkey@0"
I20260810 00:59:57.617107 1200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:59:57.639598 1200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:59:57.645777 1200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:59:57.646253 1200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:59:57.688923 1200 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:43837
I20260810 00:59:57.689103 1271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:43837 every 8 connection(s)
I20260810 00:59:57.691010 1200 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 00:59:57.696810 1272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:59:57.698964 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 1200
I20260810 00:59:57.699229 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 00:59:57.700203 20806 external_mini_cluster.cc:1428] Setting key fa5aad615f9cccd32fbb943887b71c3e
I20260810 00:59:57.718681 1272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1: Bootstrap starting.
I20260810 00:59:57.724684 1272 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1: Neither blocks nor log segments found. Creating new log.
I20260810 00:59:57.726847 1272 log.cc:824] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1: Log is configured to *not* fsync() on all Append() calls
I20260810 00:59:57.731482 1272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1: No bootstrap required, opened a new log
Aug 10 00:59:57 dist-test-slave-gp6n krb5kdc[699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 00:59:57.746804 1272 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7d268f14f84181ab248f3021cb52b1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43837 } }
I20260810 00:59:57.747399 1272 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:59:57.747524 1272 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c7d268f14f84181ab248f3021cb52b1, State: Initialized, Role: FOLLOWER
I20260810 00:59:57.748440 1278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:59:57.702663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32810 (local address 127.20.81.190:43837)
0810 00:59:57.718079 (+ 15416us) server_negotiation.cc:206] Beginning negotiation
0810 00:59:57.718099 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 00:59:57.718167 (+ 68us) server_negotiation.cc:407] Connection header received
0810 00:59:57.721814 (+ 3647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:59:57.721867 (+ 53us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:59:57.722628 (+ 761us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:59:57.723160 (+ 532us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:59:57.724811 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:57.727633 (+ 2822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:57.729523 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:59:57.730860 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:59:57.731040 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:59:57.736619 (+ 5579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:59:57.736631 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:59:57.736650 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:59:57.736716 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:59:57.742457 (+ 5741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:59:57.743325 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:59:57.743342 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:59:57.743351 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:59:57.743626 (+ 275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:59:57.744095 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:59:57.744102 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:59:57.744106 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:59:57.745184 (+ 1078us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:59:57.745635 (+ 451us) server_negotiation.cc:1035] Waiting for connection context
0810 00:59:57.746040 (+ 405us) server_negotiation.cc:299] Negotiation successful
0810 00:59:57.746576 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15085,"thread_start_us":14700,"threads_started":1}
I20260810 00:59:57.748489 1272 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7d268f14f84181ab248f3021cb52b1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43837 } }
I20260810 00:59:57.748942 1272 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:59:57.749164 1272 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:59:57.749403 1272 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:59:57.753291 1272 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7d268f14f84181ab248f3021cb52b1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43837 } }
I20260810 00:59:57.753896 1272 leader_election.cc:302] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c7d268f14f84181ab248f3021cb52b1; no voters:
I20260810 00:59:57.755157 1272 leader_election.cc:288] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:59:57.755316 1282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:59:57.756757 1282 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1 [term 1 LEADER]: Becoming Leader. State: Replica: 1c7d268f14f84181ab248f3021cb52b1, State: Running, Role: LEADER
I20260810 00:59:57.757907 1282 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7d268f14f84181ab248f3021cb52b1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43837 } }
I20260810 00:59:57.761971 1282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c7d268f14f84181ab248f3021cb52b1. Latest consensus state: current_term: 1 leader_uuid: "1c7d268f14f84181ab248f3021cb52b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7d268f14f84181ab248f3021cb52b1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43837 } } }
I20260810 00:59:57.764737 1282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1 [sys.catalog]: This master's current role is: LEADER
I20260810 00:59:57.764321 1284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c7d268f14f84181ab248f3021cb52b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7d268f14f84181ab248f3021cb52b1" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43837 } } }
I20260810 00:59:57.766194 1284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1 [sys.catalog]: This master's current role is: LEADER
I20260810 00:59:57.766842 1272 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:59:57.768365 1286 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:59:57.783239 1286 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:59:57.813333 1286 catalog_manager.cc:1347] Generated new cluster ID: 06e8e7759c5e46dc8a34350330ab4996
I20260810 00:59:57.813606 1286 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:59:57.832178 1286 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:59:57.834451 1286 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:59:57.849963 1272 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 00:59:57.854225 1286 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1: Generated new TSK 0
I20260810 00:59:57.866593 1286 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.81.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
I20260810 00:59:58.954818 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.81.148:58115/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:43837
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:41343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 00:59:59 dist-test-slave-gp6n krb5kdc[699](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786323599, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 00:59:59.646986 1325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:59:59.647464 1325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:59:59.647776 1325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:59:59.659232 1325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:59:59.659495 1325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:59:59.659760 1325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 00:59:59.674819 1325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:41343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.20.81.148:58115/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:43837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:59:59.676656 1325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:59:59.678295 1325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:59:59.702790 1335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:59.706140 1336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:59:59.708516 1339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:59:59.710031 1325 server_base.cc:1034] running on GCE node
I20260810 00:59:59.710951 1325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:59:59.712266 1325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:59:59.713584 1325 hybrid_clock.cc:648] HybridClock initialized: now 1786323599713407 us; error 100 us; skew 500 ppm
I20260810 00:59:59.714020 1325 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:59:59 dist-test-slave-gp6n krb5kdc[699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323599, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:59:59.730396 1325 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 00:59:59.736980 1325 webserver.cc:460] Webserver started at http://127.20.81.129:33761/ using document root <none> and password file <none>
I20260810 00:59:59.738584 1325 fs_manager.cc:359] Metadata directory not provided
I20260810 00:59:59.739042 1325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:59:59.739789 1325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 00:59:59 dist-test-slave-gp6n krb5kdc[699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323599, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
I20260810 00:59:59.768889 1325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "79bfe0b17aef4f2ab41d837c41b3859c"
format_stamp: "Formatted at 2026-08-10 00:59:59 on dist-test-slave-gp6n"
server_key: "68231a6b7c589fd5014670c4885b8ea8"
server_key_iv: "d2019b49d87e3e5458bb7b00fa019414"
server_key_version: "kuduclusterkey@0"
I20260810 00:59:59.770630 1325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "79bfe0b17aef4f2ab41d837c41b3859c"
format_stamp: "Formatted at 2026-08-10 00:59:59 on dist-test-slave-gp6n"
server_key: "68231a6b7c589fd5014670c4885b8ea8"
server_key_iv: "d2019b49d87e3e5458bb7b00fa019414"
server_key_version: "kuduclusterkey@0"
I20260810 00:59:59.781780 1325 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.008s sys 0.000s
I20260810 00:59:59.820158 1346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:59:59.822764 1325 fs_manager.cc:725] Time spent opening block manager: real 0.019s user 0.005s sys 0.012s
I20260810 00:59:59.823071 1325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "79bfe0b17aef4f2ab41d837c41b3859c"
format_stamp: "Formatted at 2026-08-10 00:59:59 on dist-test-slave-gp6n"
server_key: "68231a6b7c589fd5014670c4885b8ea8"
server_key_iv: "d2019b49d87e3e5458bb7b00fa019414"
server_key_version: "kuduclusterkey@0"
I20260810 00:59:59.823462 1325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:59:59.883052 1325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:59:59.891989 1325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:59:59.892550 1325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:59:59.911165 1325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:59:59.924307 1325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:59:59.924666 1325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.001s
I20260810 00:59:59.925082 1325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:59:59.925346 1325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 00:59:59 dist-test-slave-gp6n krb5kdc[699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323599, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
Aug 10 01:00:00 dist-test-slave-gp6n krb5kdc[699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323599, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
I20260810 01:00:00.640606 1325 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:40707
I20260810 01:00:00.643473 1325 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 01:00:00.640733 1478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:40707 every 8 connection(s)
I20260810 01:00:00.653781 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 1325
I20260810 01:00:00.654021 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 01:00:00.655159 20806 external_mini_cluster.cc:1428] Setting key 420930415672b5ff2b6c5aeea271a482
Aug 10 01:00:00 dist-test-slave-gp6n krb5kdc[699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323599, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:00:00.722005 1481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:00.658722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:46279 (local address 127.20.81.190:43837)
0810 01:00:00.687858 (+ 29136us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:00.687864 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:00.687897 (+ 33us) server_negotiation.cc:407] Connection header received
0810 01:00:00.694184 (+ 6287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:00.694192 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:00.694300 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:00.694456 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:00:00.699703 (+ 5247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:00.701248 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:00.704186 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:00.705170 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:00.705275 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:00.712105 (+ 6830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:00:00.712116 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:00:00.712123 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:00:00.712172 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:00:00.717149 (+ 4977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:00.718421 (+ 1272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:00.718427 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:00.718431 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:00.718693 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:00.719326 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:00.719332 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:00.719336 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:00.719769 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:00:00.719903 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:00.721052 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0810 01:00:00.721353 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28984,"thread_start_us":29149,"threads_started":1}
I20260810 01:00:00.724813 1482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:00.659870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:43837 (local address 127.20.81.129:46279)
0810 01:00:00.684507 (+ 24637us) negotiation.cc:107] Waiting for socket to connect
0810 01:00:00.684588 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0810 01:00:00.692087 (+ 7499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:00:00.695076 (+ 2989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:00:00.695091 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:00:00.695770 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:00:00.696768 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:00.696803 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:00.701638 (+ 4835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:00.701644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:00.703946 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:00.704011 (+ 65us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:00.705840 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:00.705849 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:00.706505 (+ 656us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:00.707637 (+ 1132us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:00:00.707669 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:00:00.711724 (+ 4055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:00:00.717362 (+ 5638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:00:00.717371 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:00:00.717391 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:00:00.718236 (+ 845us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:00:00.718859 (+ 623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:00:00.718863 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:00:00.718867 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:00:00.719169 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:00:00.720047 (+ 878us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:00:00.720056 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:00:00.720550 (+ 494us) client_negotiation.cc:769] Sending connection context
0810 01:00:00.722381 (+ 1831us) client_negotiation.cc:240] Negotiation successful
0810 01:00:00.723038 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24162,"thread_start_us":26380,"threads_started":1}
I20260810 01:00:00.727006 1479 heartbeater.cc:347] Connected to a master server at 127.20.81.190:43837
I20260810 01:00:00.727373 1479 heartbeater.cc:464] Registering TS with master...
I20260810 01:00:00.728796 1479 heartbeater.cc:511] Master 127.20.81.190:43837 requested a full tablet report, sending...
I20260810 01:00:00.731962 1233 ts_manager.cc:194] Registered new tserver with Master: 79bfe0b17aef4f2ab41d837c41b3859c (127.20.81.129:40707)
I20260810 01:00:00.734356 1233 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:46279
WARNING: no policy specified for kudu/127.20.81.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
W20260810 01:00:00.798594 20806 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.20.81.130: real 0.140s user 0.005s sys 0.019s
I20260810 01:00:00.799666 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.81.148:58115/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:43837
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:41343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:00:01.451785 1485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:00:01.452059 1485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:00:01.452212 1485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:00:01.460244 1485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:00:01.460362 1485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:00:01.460542 1485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 01:00:01.470602 1485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:41343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.20.81.148:58115/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:43837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:00:01.471909 1485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:00:01.473258 1485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:00:01.489738 1499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:00:01.489991 1498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:00:01.490154 1501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:00:01.490839 1485 server_base.cc:1034] running on GCE node
I20260810 01:00:01.491456 1485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:00:01.492496 1485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:00:01.493757 1485 hybrid_clock.cc:648] HybridClock initialized: now 1786323601493692 us; error 58 us; skew 500 ppm
I20260810 01:00:01.494132 1485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:00:01 dist-test-slave-gp6n krb5kdc[699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323601, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:00:01.500214 1485 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 01:00:01.503086 1485 webserver.cc:460] Webserver started at http://127.20.81.130:36795/ using document root <none> and password file <none>
I20260810 01:00:01.504259 1485 fs_manager.cc:359] Metadata directory not provided
I20260810 01:00:01.504442 1485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:00:01.504841 1485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 01:00:01 dist-test-slave-gp6n krb5kdc[699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323601, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
I20260810 01:00:01.528124 1485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "b555fe4c743c4d4e8a7ef115e77af28e"
format_stamp: "Formatted at 2026-08-10 01:00:01 on dist-test-slave-gp6n"
server_key: "2f85d5bc902ef6686f720947da011494"
server_key_iv: "9d6f1629f7c4c1b84388e4360906dde1"
server_key_version: "kuduclusterkey@0"
I20260810 01:00:01.529644 1485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "b555fe4c743c4d4e8a7ef115e77af28e"
format_stamp: "Formatted at 2026-08-10 01:00:01 on dist-test-slave-gp6n"
server_key: "2f85d5bc902ef6686f720947da011494"
server_key_iv: "9d6f1629f7c4c1b84388e4360906dde1"
server_key_version: "kuduclusterkey@0"
I20260810 01:00:01.537907 1485 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.005s
I20260810 01:00:01.559926 1509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:01.561823 1485 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260810 01:00:01.562108 1485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "b555fe4c743c4d4e8a7ef115e77af28e"
format_stamp: "Formatted at 2026-08-10 01:00:01 on dist-test-slave-gp6n"
server_key: "2f85d5bc902ef6686f720947da011494"
server_key_iv: "9d6f1629f7c4c1b84388e4360906dde1"
server_key_version: "kuduclusterkey@0"
I20260810 01:00:01.562484 1485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:00:01.584757 1485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:00:01.590333 1485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:00:01.590792 1485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:00:01.592434 1485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:00:01.594707 1485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:00:01.594851 1485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:01.595045 1485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:00:01.595187 1485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:01.684321 1485 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:34587
I20260810 01:00:01.684391 1639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:34587 every 8 connection(s)
I20260810 01:00:01.686643 1485 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 01:00:01.690984 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 1485
I20260810 01:00:01.691183 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 01:00:01.691849 20806 external_mini_cluster.cc:1428] Setting key 05afff96ba04dc424558236df02b3ebe
Aug 10 01:00:01 dist-test-slave-gp6n krb5kdc[699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323601, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.81.131@KRBTEST.COM; defaulting to no policy
I20260810 01:00:01.735980 1643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:01.692731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:33541 (local address 127.20.81.190:43837)
0810 01:00:01.711029 (+ 18298us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:01.711033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:01.712010 (+ 977us) server_negotiation.cc:407] Connection header received
0810 01:00:01.715947 (+ 3937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:01.715955 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:01.716055 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:01.716194 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:00:01.719052 (+ 2858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:01.720762 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:01.723006 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:01.723627 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:01.723775 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:01.728338 (+ 4563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:00:01.728343 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:00:01.728346 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:00:01.728379 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:00:01.732212 (+ 3833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:01.733126 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:01.733131 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:01.733133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:01.733312 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:01.733829 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:01.733833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:01.733835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:01.734176 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:00:01.734338 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:01.735171 (+ 833us) server_negotiation.cc:299] Negotiation successful
0810 01:00:01.735440 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18135,"thread_start_us":19305,"threads_started":1}
I20260810 01:00:01.737072 1642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:01.691887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:43837 (local address 127.20.81.130:33541)
0810 01:00:01.711821 (+ 19934us) negotiation.cc:107] Waiting for socket to connect
0810 01:00:01.711863 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 01:00:01.715521 (+ 3658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:00:01.716547 (+ 1026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:00:01.716558 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:00:01.717514 (+ 956us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:00:01.718840 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:01.718867 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:01.720916 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:01.720920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:01.722833 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:01.722853 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:01.723905 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:01.723914 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:01.724318 (+ 404us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:01.725181 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:00:01.725205 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:00:01.728079 (+ 2874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:00:01.732358 (+ 4279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:00:01.732366 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:00:01.732384 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:00:01.732988 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:00:01.733431 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:00:01.733437 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:00:01.733440 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:00:01.733698 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:00:01.734318 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:00:01.734326 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:00:01.734731 (+ 405us) client_negotiation.cc:769] Sending connection context
0810 01:00:01.735013 (+ 282us) client_negotiation.cc:240] Negotiation successful
0810 01:00:01.735665 (+ 652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19670,"thread_start_us":19418,"threads_started":1}
I20260810 01:00:01.738714 1479 heartbeater.cc:503] Master 127.20.81.190:43837 was elected leader, sending a full tablet report...
I20260810 01:00:01.739145 1640 heartbeater.cc:347] Connected to a master server at 127.20.81.190:43837
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.131@KRBTEST.COM" created.
I20260810 01:00:01.739573 1640 heartbeater.cc:464] Registering TS with master...
I20260810 01:00:01.740720 1640 heartbeater.cc:511] Master 127.20.81.190:43837 requested a full tablet report, sending...
I20260810 01:00:01.742651 1233 ts_manager.cc:194] Registered new tserver with Master: b555fe4c743c4d4e8a7ef115e77af28e (127.20.81.130:34587)
I20260810 01:00:01.743535 1233 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:33541
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
I20260810 01:00:01.774699 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.81.148:58115/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:43837
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:41343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:00:02.427791 1647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:00:02.428160 1647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:00:02.428357 1647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:00:02.438809 1647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:00:02.438984 1647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:00:02.439174 1647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 01:00:02.452872 1647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:41343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.20.81.148:58115/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:43837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:00:02.454322 1647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:00:02.455888 1647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:00:02.476950 1652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:00:02.476981 1655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:00:02.477123 1653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:00:02.479624 1647 server_base.cc:1034] running on GCE node
I20260810 01:00:02.480365 1647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:00:02.481601 1647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:00:02.482836 1647 hybrid_clock.cc:648] HybridClock initialized: now 1786323602482741 us; error 105 us; skew 500 ppm
I20260810 01:00:02.483202 1647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:00:02 dist-test-slave-gp6n krb5kdc[699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323602, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:00:02.488046 1647 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 01:00:02.490243 1647 webserver.cc:460] Webserver started at http://127.20.81.131:34843/ using document root <none> and password file <none>
I20260810 01:00:02.491294 1647 fs_manager.cc:359] Metadata directory not provided
I20260810 01:00:02.491533 1647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:00:02.492024 1647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 01:00:02 dist-test-slave-gp6n krb5kdc[699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323602, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
I20260810 01:00:02.511227 1647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "9d200bac560841ad920ce81ebfb1c5eb"
format_stamp: "Formatted at 2026-08-10 01:00:02 on dist-test-slave-gp6n"
server_key: "63304f7ee1612797ab8d15073eb064a5"
server_key_iv: "4d3b892ad650a020b61c9c1ae592585e"
server_key_version: "kuduclusterkey@0"
I20260810 01:00:02.512306 1647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "9d200bac560841ad920ce81ebfb1c5eb"
format_stamp: "Formatted at 2026-08-10 01:00:02 on dist-test-slave-gp6n"
server_key: "63304f7ee1612797ab8d15073eb064a5"
server_key_iv: "4d3b892ad650a020b61c9c1ae592585e"
server_key_version: "kuduclusterkey@0"
I20260810 01:00:02.519124 1647 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260810 01:00:02.538903 1662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:02.541078 1647 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260810 01:00:02.541388 1647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "9d200bac560841ad920ce81ebfb1c5eb"
format_stamp: "Formatted at 2026-08-10 01:00:02 on dist-test-slave-gp6n"
server_key: "63304f7ee1612797ab8d15073eb064a5"
server_key_iv: "4d3b892ad650a020b61c9c1ae592585e"
server_key_version: "kuduclusterkey@0"
I20260810 01:00:02.541762 1647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:00:02.562672 1647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:00:02.568387 1647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:00:02.569017 1647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:00:02.571183 1647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:00:02.574652 1647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:00:02.574862 1647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:02.575139 1647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:00:02.575240 1647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:02.667145 1647 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:43621
I20260810 01:00:02.667202 1792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:43621 every 8 connection(s)
I20260810 01:00:02.669090 1647 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 01:00:02.673609 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 1647
I20260810 01:00:02.673825 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 01:00:02.674391 20806 external_mini_cluster.cc:1428] Setting key 491a6554cb4b0dbd81a73f2d149a4e8f
Aug 10 01:00:02 dist-test-slave-gp6n krb5kdc[699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323602, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:00:02.703994 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:02.672764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:34663 (local address 127.20.81.190:43837)
0810 01:00:02.674004 (+ 1240us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:02.674014 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:02.675623 (+ 1609us) server_negotiation.cc:407] Connection header received
0810 01:00:02.679251 (+ 3628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:02.679258 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:02.679430 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:02.679584 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:00:02.681513 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:02.683145 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:02.686239 (+ 3094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:02.686943 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:02.687054 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:02.692889 (+ 5835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:00:02.692896 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:00:02.692901 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:00:02.692983 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:00:02.699173 (+ 6190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:02.700375 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:02.700383 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:02.700387 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:02.700538 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:02.701179 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:02.701188 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:02.701192 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:02.701685 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:00:02.701848 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:02.702976 (+ 1128us) server_negotiation.cc:299] Negotiation successful
0810 01:00:02.703306 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1061,"thread_start_us":726,"threads_started":1}
I20260810 01:00:02.705421 1798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:02.673450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:43837 (local address 127.20.81.131:34663)
0810 01:00:02.675424 (+ 1974us) negotiation.cc:107] Waiting for socket to connect
0810 01:00:02.675473 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 01:00:02.678890 (+ 3417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:00:02.679891 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:00:02.679901 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:00:02.680634 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:00:02.681316 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:02.681335 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:02.683313 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:02.683319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:02.686015 (+ 2696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:02.686043 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:02.687333 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:02.687341 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:02.687740 (+ 399us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:02.688959 (+ 1219us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:00:02.689003 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:00:02.692613 (+ 3610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:00:02.699404 (+ 6791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:00:02.699413 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:00:02.699429 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:00:02.700222 (+ 793us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:00:02.700693 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:00:02.700697 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:00:02.700699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:00:02.701052 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:00:02.701950 (+ 898us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:00:02.701959 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:00:02.702539 (+ 580us) client_negotiation.cc:769] Sending connection context
0810 01:00:02.702888 (+ 349us) client_negotiation.cc:240] Negotiation successful
0810 01:00:02.703561 (+ 673us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1575,"thread_start_us":1084,"threads_started":1}
I20260810 01:00:02.707414 1793 heartbeater.cc:347] Connected to a master server at 127.20.81.190:43837
I20260810 01:00:02.707790 1793 heartbeater.cc:464] Registering TS with master...
I20260810 01:00:02.708897 1793 heartbeater.cc:511] Master 127.20.81.190:43837 requested a full tablet report, sending...
I20260810 01:00:02.711256 1233 ts_manager.cc:194] Registered new tserver with Master: 9d200bac560841ad920ce81ebfb1c5eb (127.20.81.131:43621)
I20260810 01:00:02.712234 1233 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:34663
I20260810 01:00:02.718608 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 01:00:02.746506 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:02.725655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36674 (local address 127.20.81.190:43837)
0810 01:00:02.725877 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:02.725881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:02.726889 (+ 1008us) server_negotiation.cc:407] Connection header received
0810 01:00:02.727092 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:02.727096 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:02.727219 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:02.727311 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:00:02.730271 (+ 2960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:02.731415 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:02.733305 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:02.734078 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:02.734182 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:02.738207 (+ 4025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:00:02.738212 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:00:02.738216 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:00:02.738252 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:00:02.742962 (+ 4710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:02.744046 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:02.744050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:02.744052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:02.744221 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:02.744723 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:02.744727 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:02.744729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:02.745096 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:00:02.745348 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:02.745832 (+ 484us) server_negotiation.cc:299] Negotiation successful
0810 01:00:02.746053 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 01:00:02.747623 1640 heartbeater.cc:503] Master 127.20.81.190:43837 was elected leader, sending a full tablet report...
I20260810 01:00:02.751955 1233 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36674:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 01:00:02.764989 1233 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 01:00:02.805519 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:02.784874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43162 (local address 127.20.81.129:40707)
0810 01:00:02.787511 (+ 2637us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:02.787526 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:02.790004 (+ 2478us) server_negotiation.cc:407] Connection header received
0810 01:00:02.790669 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:02.790682 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:02.791088 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:02.791391 (+ 303us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:00:02.794012 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:02.797293 (+ 3281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:02.801225 (+ 3932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:02.803243 (+ 2018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:02.803381 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:02.803749 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:02.804537 (+ 788us) server_negotiation.cc:299] Negotiation successful
0810 01:00:02.804845 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2347,"thread_start_us":2001,"threads_started":1}
I20260810 01:00:02.805560 1813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:02.784776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:40707 (local address 127.0.0.1:43162)
0810 01:00:02.789771 (+ 4995us) negotiation.cc:107] Waiting for socket to connect
0810 01:00:02.789841 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0810 01:00:02.790325 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:00:02.791648 (+ 1323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:00:02.791657 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:00:02.791712 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:00:02.793571 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:02.793597 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:02.797485 (+ 3888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:02.797491 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:02.801021 (+ 3530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:02.801034 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:02.804027 (+ 2993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:02.804037 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:02.804275 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:02.804313 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 01:00:02.804473 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 01:00:02.804753 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4540,"mutex_wait_us":23,"thread_start_us":6683,"threads_started":1}
I20260810 01:00:02.808059 1816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:02.789772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:43621 (local address 127.0.0.1:50452)
0810 01:00:02.792452 (+ 2680us) negotiation.cc:107] Waiting for socket to connect
0810 01:00:02.792471 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 01:00:02.792620 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:00:02.793969 (+ 1349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:00:02.793975 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:00:02.794000 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:00:02.795419 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:02.795445 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:02.798253 (+ 2808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:02.798260 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:02.803099 (+ 4839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:02.803114 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:02.806620 (+ 3506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:02.806627 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:02.806808 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:02.806831 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 01:00:02.806933 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 01:00:02.807069 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2556,"mutex_wait_us":119,"thread_start_us":1542,"threads_started":1}
I20260810 01:00:02.808035 1817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:02.790260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50452 (local address 127.20.81.131:43621)
0810 01:00:02.792509 (+ 2249us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:02.792518 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:02.792585 (+ 67us) server_negotiation.cc:407] Connection header received
0810 01:00:02.792775 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:02.792783 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:02.793119 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:02.793375 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:00:02.796058 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:02.798090 (+ 2032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:02.803347 (+ 5257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:02.805763 (+ 2416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:02.805881 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:02.806229 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:02.807003 (+ 774us) server_negotiation.cc:299] Negotiation successful
0810 01:00:02.807277 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1931,"thread_start_us":1707,"threads_started":1}
I20260810 01:00:02.810484 1814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:02.786121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58346 (local address 127.20.81.130:34587)
0810 01:00:02.788133 (+ 2012us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:02.788143 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:02.792970 (+ 4827us) server_negotiation.cc:407] Connection header received
0810 01:00:02.793389 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:02.793398 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:02.793621 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:02.793894 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:00:02.796292 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:02.799043 (+ 2751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:02.804436 (+ 5393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:02.806958 (+ 2522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:02.807133 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:02.807347 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:02.809551 (+ 2204us) server_negotiation.cc:299] Negotiation successful
0810 01:00:02.809754 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1798,"thread_start_us":1536,"threads_started":1}
I20260810 01:00:02.811558 1812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:02.785174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:34587 (local address 127.0.0.1:58346)
0810 01:00:02.792811 (+ 7637us) negotiation.cc:107] Waiting for socket to connect
0810 01:00:02.792860 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 01:00:02.793131 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:00:02.794324 (+ 1193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:00:02.794327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:00:02.794348 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:00:02.796010 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:02.796024 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:02.799257 (+ 3233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:02.799262 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:02.804263 (+ 5001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:02.804277 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:02.807711 (+ 3434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:02.807719 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:02.808042 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:02.808076 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 01:00:02.808188 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 01:00:02.808321 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6594,"mutex_wait_us":76,"thread_start_us":3355,"threads_started":1}
I20260810 01:00:02.811914 1401 tablet_service.cc:1467] Processing CreateTablet for tablet 452de939683a450b8b4b6ce92c7eb8ea (DEFAULT_TABLE table=test-table [id=3e32ca90798b48ee979163785ecb264a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:00:02.812696 1717 tablet_service.cc:1467] Processing CreateTablet for tablet 452de939683a450b8b4b6ce92c7eb8ea (DEFAULT_TABLE table=test-table [id=3e32ca90798b48ee979163785ecb264a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:00:02.814018 1401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 452de939683a450b8b4b6ce92c7eb8ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:00:02.814999 1717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 452de939683a450b8b4b6ce92c7eb8ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:00:02.816699 1562 tablet_service.cc:1467] Processing CreateTablet for tablet 452de939683a450b8b4b6ce92c7eb8ea (DEFAULT_TABLE table=test-table [id=3e32ca90798b48ee979163785ecb264a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:00:02.818729 1562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 452de939683a450b8b4b6ce92c7eb8ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:00:02.834913 1818 tablet_bootstrap.cc:492] T 452de939683a450b8b4b6ce92c7eb8ea P 79bfe0b17aef4f2ab41d837c41b3859c: Bootstrap starting.
I20260810 01:00:02.838963 1819 tablet_bootstrap.cc:492] T 452de939683a450b8b4b6ce92c7eb8ea P 9d200bac560841ad920ce81ebfb1c5eb: Bootstrap starting.
I20260810 01:00:02.842978 1820 tablet_bootstrap.cc:492] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e: Bootstrap starting.
I20260810 01:00:02.845373 1818 tablet_bootstrap.cc:654] T 452de939683a450b8b4b6ce92c7eb8ea P 79bfe0b17aef4f2ab41d837c41b3859c: Neither blocks nor log segments found. Creating new log.
I20260810 01:00:02.846845 1819 tablet_bootstrap.cc:654] T 452de939683a450b8b4b6ce92c7eb8ea P 9d200bac560841ad920ce81ebfb1c5eb: Neither blocks nor log segments found. Creating new log.
I20260810 01:00:02.848119 1820 tablet_bootstrap.cc:654] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e: Neither blocks nor log segments found. Creating new log.
I20260810 01:00:02.848618 1818 log.cc:824] T 452de939683a450b8b4b6ce92c7eb8ea P 79bfe0b17aef4f2ab41d837c41b3859c: Log is configured to *not* fsync() on all Append() calls
I20260810 01:00:02.849785 1819 log.cc:824] T 452de939683a450b8b4b6ce92c7eb8ea P 9d200bac560841ad920ce81ebfb1c5eb: Log is configured to *not* fsync() on all Append() calls
I20260810 01:00:02.850531 1820 log.cc:824] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e: Log is configured to *not* fsync() on all Append() calls
I20260810 01:00:02.856421 1819 tablet_bootstrap.cc:492] T 452de939683a450b8b4b6ce92c7eb8ea P 9d200bac560841ad920ce81ebfb1c5eb: No bootstrap required, opened a new log
I20260810 01:00:02.856842 1820 tablet_bootstrap.cc:492] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e: No bootstrap required, opened a new log
I20260810 01:00:02.856895 1819 ts_tablet_manager.cc:1397] T 452de939683a450b8b4b6ce92c7eb8ea P 9d200bac560841ad920ce81ebfb1c5eb: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.002s
I20260810 01:00:02.857273 1820 ts_tablet_manager.cc:1397] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.004s
I20260810 01:00:02.857870 1818 tablet_bootstrap.cc:492] T 452de939683a450b8b4b6ce92c7eb8ea P 79bfe0b17aef4f2ab41d837c41b3859c: No bootstrap required, opened a new log
I20260810 01:00:02.858392 1818 ts_tablet_manager.cc:1397] T 452de939683a450b8b4b6ce92c7eb8ea P 79bfe0b17aef4f2ab41d837c41b3859c: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.014s
I20260810 01:00:02.869567 1820 raft_consensus.cc:348] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bfe0b17aef4f2ab41d837c41b3859c" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40707 } } peers { permanent_uuid: "b555fe4c743c4d4e8a7ef115e77af28e" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 34587 } } peers { permanent_uuid: "9d200bac560841ad920ce81ebfb1c5eb" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43621 } }
I20260810 01:00:02.870461 1820 raft_consensus.cc:374] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:00:02.870700 1820 raft_consensus.cc:729] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b555fe4c743c4d4e8a7ef115e77af28e, State: Initialized, Role: FOLLOWER
I20260810 01:00:02.871537 1820 consensus_queue.cc:260] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bfe0b17aef4f2ab41d837c41b3859c" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40707 } } peers { permanent_uuid: "b555fe4c743c4d4e8a7ef115e77af28e" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 34587 } } peers { permanent_uuid: "9d200bac560841ad920ce81ebfb1c5eb" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43621 } }
I20260810 01:00:02.874018 1820 ts_tablet_manager.cc:1428] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e: Time spent starting tablet: real 0.017s user 0.009s sys 0.009s
I20260810 01:00:02.877199 1818 raft_consensus.cc:348] T 452de939683a450b8b4b6ce92c7eb8ea P 79bfe0b17aef4f2ab41d837c41b3859c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bfe0b17aef4f2ab41d837c41b3859c" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40707 } } peers { permanent_uuid: "b555fe4c743c4d4e8a7ef115e77af28e" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 34587 } } peers { permanent_uuid: "9d200bac560841ad920ce81ebfb1c5eb" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43621 } }
I20260810 01:00:02.877173 1819 raft_consensus.cc:348] T 452de939683a450b8b4b6ce92c7eb8ea P 9d200bac560841ad920ce81ebfb1c5eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bfe0b17aef4f2ab41d837c41b3859c" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40707 } } peers { permanent_uuid: "b555fe4c743c4d4e8a7ef115e77af28e" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 34587 } } peers { permanent_uuid: "9d200bac560841ad920ce81ebfb1c5eb" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43621 } }
I20260810 01:00:02.877841 1818 raft_consensus.cc:374] T 452de939683a450b8b4b6ce92c7eb8ea P 79bfe0b17aef4f2ab41d837c41b3859c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:00:02.877923 1819 raft_consensus.cc:374] T 452de939683a450b8b4b6ce92c7eb8ea P 9d200bac560841ad920ce81ebfb1c5eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:00:02.878041 1818 raft_consensus.cc:729] T 452de939683a450b8b4b6ce92c7eb8ea P 79bfe0b17aef4f2ab41d837c41b3859c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79bfe0b17aef4f2ab41d837c41b3859c, State: Initialized, Role: FOLLOWER
I20260810 01:00:02.878118 1819 raft_consensus.cc:729] T 452de939683a450b8b4b6ce92c7eb8ea P 9d200bac560841ad920ce81ebfb1c5eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d200bac560841ad920ce81ebfb1c5eb, State: Initialized, Role: FOLLOWER
I20260810 01:00:02.878962 1819 consensus_queue.cc:260] T 452de939683a450b8b4b6ce92c7eb8ea P 9d200bac560841ad920ce81ebfb1c5eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bfe0b17aef4f2ab41d837c41b3859c" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40707 } } peers { permanent_uuid: "b555fe4c743c4d4e8a7ef115e77af28e" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 34587 } } peers { permanent_uuid: "9d200bac560841ad920ce81ebfb1c5eb" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43621 } }
I20260810 01:00:02.879069 1818 consensus_queue.cc:260] T 452de939683a450b8b4b6ce92c7eb8ea P 79bfe0b17aef4f2ab41d837c41b3859c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bfe0b17aef4f2ab41d837c41b3859c" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40707 } } peers { permanent_uuid: "b555fe4c743c4d4e8a7ef115e77af28e" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 34587 } } peers { permanent_uuid: "9d200bac560841ad920ce81ebfb1c5eb" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43621 } }
I20260810 01:00:02.881533 1793 heartbeater.cc:503] Master 127.20.81.190:43837 was elected leader, sending a full tablet report...
I20260810 01:00:02.882112 1819 ts_tablet_manager.cc:1428] T 452de939683a450b8b4b6ce92c7eb8ea P 9d200bac560841ad920ce81ebfb1c5eb: Time spent starting tablet: real 0.025s user 0.021s sys 0.004s
I20260810 01:00:02.884958 1818 ts_tablet_manager.cc:1428] T 452de939683a450b8b4b6ce92c7eb8ea P 79bfe0b17aef4f2ab41d837c41b3859c: Time spent starting tablet: real 0.026s user 0.012s sys 0.013s
W20260810 01:00:02.922473 1796 tablet.cc:2369] T 452de939683a450b8b4b6ce92c7eb8ea P 9d200bac560841ad920ce81ebfb1c5eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 01:00:02.950273 1480 tablet.cc:2369] T 452de939683a450b8b4b6ce92c7eb8ea P 79bfe0b17aef4f2ab41d837c41b3859c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 01:00:02.960953 1641 tablet.cc:2369] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:00:03.022219 1824 raft_consensus.cc:482] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:00:03.022682 1824 raft_consensus.cc:504] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bfe0b17aef4f2ab41d837c41b3859c" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40707 } } peers { permanent_uuid: "b555fe4c743c4d4e8a7ef115e77af28e" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 34587 } } peers { permanent_uuid: "9d200bac560841ad920ce81ebfb1c5eb" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43621 } }
I20260810 01:00:03.024624 1824 leader_election.cc:288] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79bfe0b17aef4f2ab41d837c41b3859c (127.20.81.129:40707), 9d200bac560841ad920ce81ebfb1c5eb (127.20.81.131:43621)
I20260810 01:00:03.036283 1817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:03.025601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:42835 (local address 127.20.81.131:43621)
0810 01:00:03.025892 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:03.025900 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:03.027955 (+ 2055us) server_negotiation.cc:407] Connection header received
0810 01:00:03.028279 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:03.028285 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:03.028370 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:03.028513 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:00:03.029401 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:03.030662 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:03.033252 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:03.034817 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:03.035028 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:03.035100 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:03.035785 (+ 685us) server_negotiation.cc:299] Negotiation successful
0810 01:00:03.035897 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 01:00:03.036368 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:03.025592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:43621 (local address 127.20.81.130:42835)
0810 01:00:03.027770 (+ 2178us) negotiation.cc:107] Waiting for socket to connect
0810 01:00:03.027818 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 01:00:03.027977 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:00:03.028529 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:00:03.028533 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:00:03.028558 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:00:03.029154 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:03.029169 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:03.030809 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:03.030815 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:03.032971 (+ 2156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:03.032986 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:03.035144 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:03.035151 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:03.035388 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:03.035445 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 01:00:03.035567 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 01:00:03.035755 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2077,"mutex_wait_us":24,"thread_start_us":1677,"threads_started":1}
I20260810 01:00:03.037281 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:03.025059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:53545 (local address 127.20.81.129:40707)
0810 01:00:03.025272 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:03.025277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:03.027846 (+ 2569us) server_negotiation.cc:407] Connection header received
0810 01:00:03.028129 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:03.028136 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:03.028312 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:03.028471 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:00:03.029661 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:03.031487 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:03.034013 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:03.035981 (+ 1968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:03.036244 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:03.036356 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:03.036789 (+ 433us) server_negotiation.cc:299] Negotiation successful
0810 01:00:03.036965 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 01:00:03.037932 1827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:03.024977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:40707 (local address 127.20.81.130:53545)
0810 01:00:03.027703 (+ 2726us) negotiation.cc:107] Waiting for socket to connect
0810 01:00:03.027722 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 01:00:03.027946 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:00:03.028695 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:00:03.028701 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:00:03.028749 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:00:03.029407 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:03.029421 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:03.031669 (+ 2248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:03.031673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:03.033750 (+ 2077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:03.033766 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:03.036344 (+ 2578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:03.036358 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:03.036545 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:03.036576 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 01:00:03.037468 (+ 892us) client_negotiation.cc:240] Negotiation successful
0810 01:00:03.037633 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2526,"thread_start_us":1559,"threads_started":1}
I20260810 01:00:03.038353 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "452de939683a450b8b4b6ce92c7eb8ea" candidate_uuid: "b555fe4c743c4d4e8a7ef115e77af28e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d200bac560841ad920ce81ebfb1c5eb" is_pre_election: true
I20260810 01:00:03.039152 1743 raft_consensus.cc:2454] T 452de939683a450b8b4b6ce92c7eb8ea P 9d200bac560841ad920ce81ebfb1c5eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b555fe4c743c4d4e8a7ef115e77af28e in term 0.
I20260810 01:00:03.038923 1423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "452de939683a450b8b4b6ce92c7eb8ea" candidate_uuid: "b555fe4c743c4d4e8a7ef115e77af28e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79bfe0b17aef4f2ab41d837c41b3859c" is_pre_election: true
I20260810 01:00:03.039875 1423 raft_consensus.cc:2454] T 452de939683a450b8b4b6ce92c7eb8ea P 79bfe0b17aef4f2ab41d837c41b3859c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b555fe4c743c4d4e8a7ef115e77af28e in term 0.
I20260810 01:00:03.040138 1514 leader_election.cc:302] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d200bac560841ad920ce81ebfb1c5eb, b555fe4c743c4d4e8a7ef115e77af28e; no voters:
I20260810 01:00:03.040822 1824 raft_consensus.cc:2781] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:00:03.041113 1824 raft_consensus.cc:482] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:00:03.041244 1824 raft_consensus.cc:3037] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:00:03.044788 1824 raft_consensus.cc:504] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bfe0b17aef4f2ab41d837c41b3859c" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40707 } } peers { permanent_uuid: "b555fe4c743c4d4e8a7ef115e77af28e" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 34587 } } peers { permanent_uuid: "9d200bac560841ad920ce81ebfb1c5eb" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43621 } }
I20260810 01:00:03.045715 1824 leader_election.cc:288] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e [CANDIDATE]: Term 1 election: Requested vote from peers 79bfe0b17aef4f2ab41d837c41b3859c (127.20.81.129:40707), 9d200bac560841ad920ce81ebfb1c5eb (127.20.81.131:43621)
I20260810 01:00:03.046295 1423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "452de939683a450b8b4b6ce92c7eb8ea" candidate_uuid: "b555fe4c743c4d4e8a7ef115e77af28e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79bfe0b17aef4f2ab41d837c41b3859c"
I20260810 01:00:03.046478 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "452de939683a450b8b4b6ce92c7eb8ea" candidate_uuid: "b555fe4c743c4d4e8a7ef115e77af28e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d200bac560841ad920ce81ebfb1c5eb"
I20260810 01:00:03.046715 1423 raft_consensus.cc:3037] T 452de939683a450b8b4b6ce92c7eb8ea P 79bfe0b17aef4f2ab41d837c41b3859c [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:00:03.046835 1743 raft_consensus.cc:3037] T 452de939683a450b8b4b6ce92c7eb8ea P 9d200bac560841ad920ce81ebfb1c5eb [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:00:03.049677 1423 raft_consensus.cc:2454] T 452de939683a450b8b4b6ce92c7eb8ea P 79bfe0b17aef4f2ab41d837c41b3859c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b555fe4c743c4d4e8a7ef115e77af28e in term 1.
I20260810 01:00:03.050217 1513 leader_election.cc:302] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79bfe0b17aef4f2ab41d837c41b3859c, b555fe4c743c4d4e8a7ef115e77af28e; no voters:
I20260810 01:00:03.050629 1824 raft_consensus.cc:2781] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:00:03.050839 1743 raft_consensus.cc:2454] T 452de939683a450b8b4b6ce92c7eb8ea P 9d200bac560841ad920ce81ebfb1c5eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b555fe4c743c4d4e8a7ef115e77af28e in term 1.
I20260810 01:00:03.051173 1824 raft_consensus.cc:686] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e [term 1 LEADER]: Becoming Leader. State: Replica: b555fe4c743c4d4e8a7ef115e77af28e, State: Running, Role: LEADER
I20260810 01:00:03.051831 1824 consensus_queue.cc:237] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bfe0b17aef4f2ab41d837c41b3859c" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40707 } } peers { permanent_uuid: "b555fe4c743c4d4e8a7ef115e77af28e" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 34587 } } peers { permanent_uuid: "9d200bac560841ad920ce81ebfb1c5eb" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43621 } }
I20260810 01:00:03.057603 1231 catalog_manager.cc:5495] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e reported cstate change: term changed from 0 to 1, leader changed from <none> to b555fe4c743c4d4e8a7ef115e77af28e (127.20.81.130). New cstate: current_term: 1 leader_uuid: "b555fe4c743c4d4e8a7ef115e77af28e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bfe0b17aef4f2ab41d837c41b3859c" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b555fe4c743c4d4e8a7ef115e77af28e" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 34587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d200bac560841ad920ce81ebfb1c5eb" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43621 } health_report { overall_health: UNKNOWN } } }
I20260810 01:00:03.101397 1814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:03.095387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58352 (local address 127.20.81.130:34587)
0810 01:00:03.095681 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:03.095686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:03.095704 (+ 18us) server_negotiation.cc:407] Connection header received
0810 01:00:03.095863 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:03.095868 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:03.095953 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:03.096161 (+ 208us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:00:03.096597 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:03.097425 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:03.099079 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:03.099535 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:03.099679 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:03.100234 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:00:03.100769 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:00:03.100955 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:03.101013 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 01:00:03.101101 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 01:00:03.111164 1743 raft_consensus.cc:1261] T 452de939683a450b8b4b6ce92c7eb8ea P 9d200bac560841ad920ce81ebfb1c5eb [term 1 FOLLOWER]: Refusing update from remote peer b555fe4c743c4d4e8a7ef115e77af28e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:00:03.111179 1423 raft_consensus.cc:1261] T 452de939683a450b8b4b6ce92c7eb8ea P 79bfe0b17aef4f2ab41d837c41b3859c [term 1 FOLLOWER]: Refusing update from remote peer b555fe4c743c4d4e8a7ef115e77af28e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:00:03.112210 1831 consensus_queue.cc:1035] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d200bac560841ad920ce81ebfb1c5eb" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 43621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:00:03.112597 1824 consensus_queue.cc:1035] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e [LEADER]: Connected to new peer: Peer: permanent_uuid: "79bfe0b17aef4f2ab41d837c41b3859c" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:00:03.141361 1231 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:00:03.141791 1231 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:00:03.147356 1231 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c7d268f14f84181ab248f3021cb52b1: Sending DeleteTablet for 3 replicas of tablet 452de939683a450b8b4b6ce92c7eb8ea
I20260810 01:00:03.148953 1401 tablet_service.cc:1514] Processing DeleteTablet for tablet 452de939683a450b8b4b6ce92c7eb8ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:00:03 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:43162
I20260810 01:00:03.149016 1562 tablet_service.cc:1514] Processing DeleteTablet for tablet 452de939683a450b8b4b6ce92c7eb8ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:00:03 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:58346
I20260810 01:00:03.150475 1717 tablet_service.cc:1514] Processing DeleteTablet for tablet 452de939683a450b8b4b6ce92c7eb8ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:00:03 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:50452
I20260810 01:00:03.151430 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 1325
I20260810 01:00:03.152455 1849 tablet_replica.cc:330] T 452de939683a450b8b4b6ce92c7eb8ea P 9d200bac560841ad920ce81ebfb1c5eb: stopping tablet replica
I20260810 01:00:03.152962 1849 raft_consensus.cc:2229] T 452de939683a450b8b4b6ce92c7eb8ea P 9d200bac560841ad920ce81ebfb1c5eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:00:03.153888 1849 raft_consensus.cc:2258] T 452de939683a450b8b4b6ce92c7eb8ea P 9d200bac560841ad920ce81ebfb1c5eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:00:03.156791 1849 ts_tablet_manager.cc:1905] T 452de939683a450b8b4b6ce92c7eb8ea P 9d200bac560841ad920ce81ebfb1c5eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:00:03.164129 1848 tablet_replica.cc:330] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e: stopping tablet replica
I20260810 01:00:03.164978 1848 raft_consensus.cc:2229] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e [term 1 LEADER]: Raft consensus shutting down.
I20260810 01:00:03.166445 1848 raft_consensus.cc:2258] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:00:03.168849 1849 ts_tablet_manager.cc:1918] T 452de939683a450b8b4b6ce92c7eb8ea P 9d200bac560841ad920ce81ebfb1c5eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:00:03.169253 1849 log.cc:1196] T 452de939683a450b8b4b6ce92c7eb8ea P 9d200bac560841ad920ce81ebfb1c5eb: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/452de939683a450b8b4b6ce92c7eb8ea
I20260810 01:00:03.169728 1848 ts_tablet_manager.cc:1905] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:00:03.170315 1849 ts_tablet_manager.cc:1939] T 452de939683a450b8b4b6ce92c7eb8ea P 9d200bac560841ad920ce81ebfb1c5eb: Deleting consensus metadata
I20260810 01:00:03.173498 1219 catalog_manager.cc:4841] TS 9d200bac560841ad920ce81ebfb1c5eb (127.20.81.131:43621): tablet 452de939683a450b8b4b6ce92c7eb8ea (table test-table [id=3e32ca90798b48ee979163785ecb264a]) successfully deleted
I20260810 01:00:03.176376 1847 tablet_replica.cc:330] T 452de939683a450b8b4b6ce92c7eb8ea P 79bfe0b17aef4f2ab41d837c41b3859c: stopping tablet replica
I20260810 01:00:03.177132 1847 raft_consensus.cc:2229] T 452de939683a450b8b4b6ce92c7eb8ea P 79bfe0b17aef4f2ab41d837c41b3859c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:00:03.178148 1847 raft_consensus.cc:2258] T 452de939683a450b8b4b6ce92c7eb8ea P 79bfe0b17aef4f2ab41d837c41b3859c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:00:03.180639 1847 ts_tablet_manager.cc:1905] T 452de939683a450b8b4b6ce92c7eb8ea P 79bfe0b17aef4f2ab41d837c41b3859c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:00:03.181553 1848 ts_tablet_manager.cc:1918] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:00:03.181984 1848 log.cc:1196] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/452de939683a450b8b4b6ce92c7eb8ea
I20260810 01:00:03.183192 1848 ts_tablet_manager.cc:1939] T 452de939683a450b8b4b6ce92c7eb8ea P b555fe4c743c4d4e8a7ef115e77af28e: Deleting consensus metadata
I20260810 01:00:03.186012 1219 catalog_manager.cc:4841] TS b555fe4c743c4d4e8a7ef115e77af28e (127.20.81.130:34587): tablet 452de939683a450b8b4b6ce92c7eb8ea (table test-table [id=3e32ca90798b48ee979163785ecb264a]) successfully deleted
Aug 10 01:00:03 dist-test-slave-gp6n krb5kdc[699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.129@KRBTEST.COM
I20260810 01:00:03.191413 1847 ts_tablet_manager.cc:1918] T 452de939683a450b8b4b6ce92c7eb8ea P 79bfe0b17aef4f2ab41d837c41b3859c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:00:03.191716 1847 log.cc:1196] T 452de939683a450b8b4b6ce92c7eb8ea P 79bfe0b17aef4f2ab41d837c41b3859c: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323445691443-20806-0/minicluster-data/ts-0/wal/wals/452de939683a450b8b4b6ce92c7eb8ea
I20260810 01:00:03.192871 1847 ts_tablet_manager.cc:1939] T 452de939683a450b8b4b6ce92c7eb8ea P 79bfe0b17aef4f2ab41d837c41b3859c: Deleting consensus metadata
I20260810 01:00:03.195704 1218 catalog_manager.cc:4841] TS 79bfe0b17aef4f2ab41d837c41b3859c (127.20.81.129:40707): tablet 452de939683a450b8b4b6ce92c7eb8ea (table test-table [id=3e32ca90798b48ee979163785ecb264a]) successfully deleted
I20260810 01:00:03.201980 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:03.152732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43170 (local address 127.20.81.129:40707)
0810 01:00:03.152915 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:03.152921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:03.174904 (+ 21983us) server_negotiation.cc:407] Connection header received
0810 01:00:03.175195 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:03.175201 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:03.175337 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:03.175499 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:00:03.177978 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:03.179313 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:03.181325 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:03.182312 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:03.182507 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:03.191358 (+ 8851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:00:03.191388 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:00:03.191415 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:00:03.191463 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:00:03.197830 (+ 6367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:03.199233 (+ 1403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:03.199241 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:03.199244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:03.199439 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:03.200109 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:03.200112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:03.200114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:03.200653 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:00:03.200990 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:03.201373 (+ 383us) server_negotiation.cc:299] Negotiation successful
0810 01:00:03.201543 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 01:00:03.203547 1474 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:00:03.299520 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 1325
I20260810 01:00:03.323051 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 1485
Aug 10 01:00:03 dist-test-slave-gp6n krb5kdc[699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.130@KRBTEST.COM
I20260810 01:00:03.342803 1814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:03.323873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58358 (local address 127.20.81.130:34587)
0810 01:00:03.324078 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:03.324084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:03.324112 (+ 28us) server_negotiation.cc:407] Connection header received
0810 01:00:03.324252 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:03.324259 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:03.324372 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:03.324480 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:00:03.325790 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:03.327062 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:03.328503 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:03.329626 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:03.329797 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:03.334912 (+ 5115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:00:03.334934 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:00:03.334957 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:00:03.335019 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:00:03.339546 (+ 4527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:03.340261 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:03.340272 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:03.340276 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:03.340454 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:03.340895 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:03.340900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:03.340905 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:03.341443 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:00:03.341764 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:03.342054 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 01:00:03.342335 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 01:00:03.343683 1635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:00:03.466228 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 1485
I20260810 01:00:03.489043 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 1647
Aug 10 01:00:03 dist-test-slave-gp6n krb5kdc[699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.131@KRBTEST.COM
I20260810 01:00:03.506021 1817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:03.489948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50454 (local address 127.20.81.131:43621)
0810 01:00:03.490134 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:03.490138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:03.490179 (+ 41us) server_negotiation.cc:407] Connection header received
0810 01:00:03.490239 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:03.490243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:03.490378 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:03.490535 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:00:03.491897 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:03.492725 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:03.494208 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:03.494777 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:03.494892 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:03.499620 (+ 4728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:00:03.499637 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:00:03.499652 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:00:03.499693 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:00:03.503120 (+ 3427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:03.503927 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:03.503934 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:03.503937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:03.504083 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:03.504551 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:03.504555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:03.504558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:03.505010 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:00:03.505249 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:03.505540 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 01:00:03.505724 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 01:00:03.506907 1787 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:00:03.598987 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 1647
I20260810 01:00:03.619262 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 1200
I20260810 01:00:03.620607 1267 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:00:03.737356 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 1200
I20260810 01:00:03.766177 20806 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 01:00:04.111575 20806 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 01:00:04.111927 20806 mini_ranger.cc:66] Stopping Ranger...
I20260810 01:00:04.457675 20806 mini_ranger.cc:68] Stopped Ranger
2026-08-10 01:00:04.457 UTC [732] LOG: received smart shutdown request
2026-08-10 01:00:04.462 UTC [732] LOG: background worker "logical replication launcher" (PID 741) exited with exit code 1
2026-08-10 01:00:04.462 UTC [736] LOG: shutting down
2026-08-10 01:00:04.634 UTC [732] LOG: database system is shut down
2026-08-10T01:00:04Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53207 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:00:04 dist-test-slave-gp6n krb5kdc[1864](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:00:04 dist-test-slave-gp6n krb5kdc[1864](info): set up 2 sockets
Aug 10 01:00:04 dist-test-slave-gp6n krb5kdc[1864](info): commencing operation
krb5kdc: starting...
W20260810 01:00:05.970105 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:00:06 dist-test-slave-gp6n krb5kdc[1864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:00:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:00:06Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.20.81.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.20.81.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.20.81.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/rangeradmin_127.20.81.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.20.81.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/rangeradmin_127.20.81.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.20.81.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.20.81.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.20.81.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/rangerlookup_127.20.81.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.20.81.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/rangerlookup_127.20.81.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.81.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.81.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.81.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/HTTP_127.20.81.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.81.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/HTTP_127.20.81.148@KRBTEST.COM.keytab.
I20260810 01:00:06.398473 20806 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task5NFMaT/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/postgres -l logfile start
2026-08-10 01:00:09.070 UTC [1897] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 01:00:09.071 UTC [1897] LOG: listening on IPv4 address "127.20.81.148", port 43915
2026-08-10 01:00:09.074 UTC [1897] LOG: listening on Unix socket "/tmp/.s.PGSQL.43915"
2026-08-10 01:00:09.099 UTC [1900] LOG: database system was shut down at 2026-08-10 01:00:07 UTC
2026-08-10 01:00:09.102 UTC [1897] LOG: database system is ready to accept connections
I20260810 01:00:10.140748 20806 mini_postgres.cc:96] Postgres bound to 43915
2026-08-10 01:00:10.164 UTC [1908] FATAL: database "slave" does not exist
127.20.81.148:43915 - accepting connections
I20260810 01:00:10.165057 20806 mini_ranger.cc:161] Starting Ranger...
I20260810 01:00:10.197544 20806 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 01:00:10.734987 20806 mini_ranger.cc:87] Created ranger Postgres database
I20260810 01:00:10.735114 20806 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ranger-admin
2026-08-10 01:00:11,055 [I] DB FLAVOR :POSTGRES
2026-08-10 01:00:11,056 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 01:00:11,056 [I] Checking connection..
2026-08-10 01:00:11,056 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 01:00:11,309 [I] Checking connection passed.
2026-08-10 01:00:11,309 [I] --------- Verifying version history table ---------
2026-08-10 01:00:11,309 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 01:00:11,559 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 01:00:11,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 01:00:11,834 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 01:00:11,834 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 01:00:11,834 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 01:00:12.098 UTC [1985] WARNING: there is no transaction in progress
2026-08-10 01:00:12,106 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 01:00:12,106 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 01:00:12,381 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 01:00:12,381 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 01:00:12,381 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 01:00:12,647 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 01:00:12,898 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gp6n.c.gcp-upstream.internal','N') ;"
2026-08-10 01:00:13,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 01:00:13,410 [I] Table x_portal_user does not exist in database ranger
2026-08-10 01:00:13,410 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 01:00:13,664 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 01:00:13,664 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 01:00:13,943 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 01:00:14,194 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 01:00:14,194 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 01:00:15.172 UTC [2107] WARNING: there is no transaction in progress
2026-08-10 01:00:15.189 UTC [2107] WARNING: there is no transaction in progress
2026-08-10 01:00:15.206 UTC [2107] WARNING: there is no transaction in progress
2026-08-10 01:00:15.223 UTC [2107] WARNING: there is no transaction in progress
2026-08-10 01:00:15.240 UTC [2107] WARNING: there is no transaction in progress
2026-08-10 01:00:15.259 UTC [2107] WARNING: there is no transaction in progress
2026-08-10 01:00:15.316 UTC [2107] WARNING: there is no transaction in progress
2026-08-10 01:00:15.322 UTC [2107] WARNING: there is no transaction in progress
2026-08-10 01:00:15.337 UTC [2107] WARNING: there is no transaction in progress
2026-08-10 01:00:15.347 UTC [2107] WARNING: there is no transaction in progress
2026-08-10 01:00:15.357 UTC [2107] WARNING: there is no transaction in progress
2026-08-10 01:00:15.371 UTC [2107] WARNING: there is no transaction in progress
2026-08-10 01:00:15.381 UTC [2107] WARNING: there is no transaction in progress
2026-08-10 01:00:15.388 UTC [2107] WARNING: there is no transaction in progress
2026-08-10 01:00:15.395 UTC [2107] WARNING: there is no transaction in progress
2026-08-10 01:00:15.402 UTC [2107] WARNING: there is no transaction in progress
2026-08-10 01:00:16,061 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 01:00:16,061 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 01:00:16,297 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 01:00:16,297 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 01:00:16,543 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 01:00:16,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 01:00:16,775 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 01:00:16,775 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 01:00:17,039 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 01:00:17,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gp6n.c.gcp-upstream.internal';"
2026-08-10 01:00:17,533 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 01:00:17,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 01:00:17,794 [I] DB_PATCHES have already been applied
I20260810 01:00:17.800648 20806 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ranger-admin:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task5NFMaT/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 01:00:17.800736 20806 mini_ranger.cc:193] Using host: 127.20.81.148
I20260810 01:00:17.821357 20806 mini_ranger.cc:227] Ranger admin URL: http://127.20.81.148:42707
Aug 10, 2026 1:00:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 1:00:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 1:00:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 1:00:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 1:00:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 1:00:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 1:00:18 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 01:00:19.128101 20806 mini_ranger.cc:160] Time spent starting Ranger: real 8.963s user 0.004s sys 0.055s
Aug 10, 2026 1:00:39 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 1:00:39 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 1:00:39 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 1:00:39 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 1:00:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 1:00:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 1:00:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 1:00:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 1:00:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 1:00:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 1:00:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 1:00:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 1:00:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 1:00:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 1:00:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 1:00:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 1:00:39 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 1:00:40 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 01:00:40.778522 20806 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.20.81.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.20.81.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.20.81.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/rangerkms_127.20.81.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.20.81.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/rangerkms_127.20.81.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.81.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.81.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.81.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/HTTP_127.20.81.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.81.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/HTTP_127.20.81.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 01:00:40 dist-test-slave-gp6n krb5kdc[1864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323640, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 01:00:40.973773 20806 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 01:00:41.000059 20806 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 01:00:41.888391 20806 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 01:00:41.888533 20806 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ranger-kms
I20260810 01:00:41.888607 20806 mini_ranger_kms.cc:227] Using postgres at 127.20.81.148:43915
2026-08-10 01:00:42,263 [I] DB FLAVOR :POSTGRES
2026-08-10 01:00:42,263 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 01:00:42,263 [I] Checking connection
2026-08-10 01:00:42,263 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 01:00:42,499 [I] connection success
2026-08-10 01:00:42,500 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 01:00:42,500 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 01:00:42,500 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 01:00:42,747 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 01:00:42,747 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 01:00:42,747 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 01:00:42,748 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.81.148:43915/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 01:00:43,001 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 01:00:43.499898 20806 mini_ranger_kms.cc:310] Created kms service
I20260810 01:00:43.690354 20806 mini_ranger_kms.cc:326] Created kudu user
I20260810 01:00:43.739710 20806 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 01:00:43 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323640, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
I20260810 01:00:44.159699 20806 mini_ranger_kms.cc:384] Added ranger policy
I20260810 01:00:44.160063 20806 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ranger-kms:/tmp/dist-test-task5NFMaT/build/asan/bin/postgresql.jar:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task5NFMaT/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 01:00:44.160125 20806 mini_ranger_kms.cc:240] Using host: 127.20.81.148
I20260810 01:00:44.188702 20806 mini_ranger_kms.cc:276] Ranger KMS PID: 2307
I20260810 01:00:44.188859 20806 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.20.81.148:46607
Aug 10, 2026 1:00:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 1:00:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 1:00:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 1:00:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 1:00:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 1:00:46 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260810 01:00:46.690536 20806 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.717s user 0.096s sys 0.311s
I20260810 01:00:46.690795 20806 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 01:00:46.691001 20806 mini_ranger_kms.cc:401] 127.20.81.148:46607/kms/v1/keys
Aug 10 01:00:46 dist-test-slave-gp6n krb5kdc[1864](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786323646, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.81.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 10 01:00:47 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323646, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.81.148@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
Aug 10 01:00:47 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323646, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.81.148@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
Aug 10 01:00:49 dist-test-slave-gp6n krb5kdc[1864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 01:00:49.895299 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:41879
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:41879
--ranger_config_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.81.148:46607/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:00:50.251277 2368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:00:50.251648 2368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:00:50.251804 2368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:00:50.259410 2368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:00:50.259575 2368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:00:50.259686 2368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:00:50.259791 2368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:00:50.259886 2368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:00:50.270932 2368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/wal
--ranger_kms_url=127.20.81.148:46607/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:41879
--ranger_config_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:41879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:00:50.272459 2368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:00:50.273978 2368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:00:50.287030 2376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:00:50.286933 2373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:00:50.287025 2374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:00:50.287693 2368 server_base.cc:1034] running on GCE node
I20260810 01:00:50.288561 2368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:00:50.290127 2368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:00:50.291326 2368 hybrid_clock.cc:648] HybridClock initialized: now 1786323650291263 us; error 62 us; skew 500 ppm
I20260810 01:00:50.291702 2368 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:00:50 dist-test-slave-gp6n krb5kdc[1864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323650, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:00:50.299775 2368 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 01:00:50.301523 2368 webserver.cc:460] Webserver started at http://127.20.81.190:40955/ using document root <none> and password file <none>
I20260810 01:00:50.302266 2368 fs_manager.cc:359] Metadata directory not provided
I20260810 01:00:50.302372 2368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:00:50.302760 2368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 01:00:50 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323650, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
I20260810 01:00:50.373342 2368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "4f78b9460c6043719a6f102ed85550c5"
format_stamp: "Formatted at 2026-08-10 01:00:50 on dist-test-slave-gp6n"
server_key: "1e22d91aacd0bf4c7e94032130587ed8"
server_key_iv: "f1f128257d96ce49681324c85531e820"
server_key_version: "kuduclusterkey@0"
I20260810 01:00:50.374639 2368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "4f78b9460c6043719a6f102ed85550c5"
format_stamp: "Formatted at 2026-08-10 01:00:50 on dist-test-slave-gp6n"
server_key: "1e22d91aacd0bf4c7e94032130587ed8"
server_key_iv: "f1f128257d96ce49681324c85531e820"
server_key_version: "kuduclusterkey@0"
I20260810 01:00:50.382273 2368 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260810 01:00:50.404197 2383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:50.406121 2368 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 01:00:50.406328 2368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "4f78b9460c6043719a6f102ed85550c5"
format_stamp: "Formatted at 2026-08-10 01:00:50 on dist-test-slave-gp6n"
server_key: "1e22d91aacd0bf4c7e94032130587ed8"
server_key_iv: "f1f128257d96ce49681324c85531e820"
server_key_version: "kuduclusterkey@0"
I20260810 01:00:50.406634 2368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:00:50.452064 2368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:00:50.456590 2368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:00:50.456939 2368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:00:50.477943 2368 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:41879
I20260810 01:00:50.478005 2443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:41879 every 8 connection(s)
I20260810 01:00:50.479439 2368 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:00:50.484331 2444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:00:50.487423 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 2368
I20260810 01:00:50.487690 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:00:50.488463 20806 external_mini_cluster.cc:1428] Setting key 3408f33086fa956654be290b1a7254f2
I20260810 01:00:50.499346 2444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5: Bootstrap starting.
I20260810 01:00:50.503597 2444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5: Neither blocks nor log segments found. Creating new log.
I20260810 01:00:50.505879 2444 log.cc:824] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5: Log is configured to *not* fsync() on all Append() calls
I20260810 01:00:50.509979 2444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5: No bootstrap required, opened a new log
Aug 10 01:00:50 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:00:50.520346 2444 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f78b9460c6043719a6f102ed85550c5" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 41879 } }
I20260810 01:00:50.520928 2444 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:00:50.521060 2444 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f78b9460c6043719a6f102ed85550c5, State: Initialized, Role: FOLLOWER
I20260810 01:00:50.521838 2444 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f78b9460c6043719a6f102ed85550c5" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 41879 } }
I20260810 01:00:50.522238 2444 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:00:50.522475 2444 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:00:50.522727 2444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:00:50.525673 2444 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f78b9460c6043719a6f102ed85550c5" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 41879 } }
I20260810 01:00:50.526300 2444 leader_election.cc:302] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f78b9460c6043719a6f102ed85550c5; no voters:
I20260810 01:00:50.527163 2444 leader_election.cc:288] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:00:50.527274 2450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:00:50.531749 2450 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [term 1 LEADER]: Becoming Leader. State: Replica: 4f78b9460c6043719a6f102ed85550c5, State: Running, Role: LEADER
I20260810 01:00:50.532394 2448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:50.491110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35334 (local address 127.20.81.190:41879)
0810 01:00:50.504327 (+ 13217us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:50.504345 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:50.504942 (+ 597us) server_negotiation.cc:407] Connection header received
0810 01:00:50.508113 (+ 3171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:50.508143 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:50.508712 (+ 569us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:50.509206 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:00:50.510947 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:50.513433 (+ 2486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:50.515421 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:50.516598 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:50.516829 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:50.522180 (+ 5351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:00:50.522190 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:00:50.522209 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:00:50.522259 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:00:50.526457 (+ 4198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:50.527393 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:50.527402 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:50.527412 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:50.527622 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:50.528101 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:50.528106 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:50.528109 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:50.528849 (+ 740us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:00:50.529133 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:50.530507 (+ 1374us) server_negotiation.cc:299] Negotiation successful
0810 01:00:50.530945 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12900,"thread_start_us":12661,"threads_started":1}
I20260810 01:00:50.532984 2450 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f78b9460c6043719a6f102ed85550c5" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 41879 } }
I20260810 01:00:50.536823 2452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f78b9460c6043719a6f102ed85550c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f78b9460c6043719a6f102ed85550c5" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 41879 } } }
I20260810 01:00:50.537247 2452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [sys.catalog]: This master's current role is: LEADER
I20260810 01:00:50.538501 2444 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:00:50.539048 2451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f78b9460c6043719a6f102ed85550c5. Latest consensus state: current_term: 1 leader_uuid: "4f78b9460c6043719a6f102ed85550c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f78b9460c6043719a6f102ed85550c5" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 41879 } } }
I20260810 01:00:50.539484 2451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [sys.catalog]: This master's current role is: LEADER
I20260810 01:00:50.540349 2454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:00:50.546912 2454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:00:50.566676 2454 catalog_manager.cc:1347] Generated new cluster ID: 91604ba4e19c4b6bb46e9301744189fc
I20260810 01:00:50.566836 2454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:00:50.581373 2454 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:00:50.583681 2454 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:00:50.596923 2444 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 01:00:50.599172 2454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5: Generated new TSK 0
I20260810 01:00:50.600360 2454 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:00:51.334434 2493 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
WARNING: no policy specified for kudu/127.20.81.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
I20260810 01:00:51.451576 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.81.148:46607/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:41879
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 01:00:51 dist-test-slave-gp6n krb5kdc[1864](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786323651, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 01:00:51.854202 2498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:00:51.854583 2498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:00:51.854770 2498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:00:51.862211 2498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:00:51.862362 2498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:00:51.862589 2498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 01:00:51.874718 2498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.20.81.148:46607/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:41879
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:00:51.876125 2498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:00:51.877478 2498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:00:51.895640 2509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:00:51.907184 2510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:00:51.908313 2498 server_base.cc:1034] running on GCE node
W20260810 01:00:51.909286 2512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:00:51.910459 2498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:00:51.912477 2498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:00:51.913849 2498 hybrid_clock.cc:648] HybridClock initialized: now 1786323651913729 us; error 107 us; skew 500 ppm
I20260810 01:00:51.914263 2498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:00:51 dist-test-slave-gp6n krb5kdc[1864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323651, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:00:51.918817 2498 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 01:00:51.923992 2498 webserver.cc:460] Webserver started at http://127.20.81.129:36357/ using document root <none> and password file <none>
I20260810 01:00:51.925037 2498 fs_manager.cc:359] Metadata directory not provided
I20260810 01:00:51.925195 2498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:00:51.925596 2498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 01:00:51 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323651, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
I20260810 01:00:51.947430 2498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "506acb14a13c419e947d7d75455b49a7"
format_stamp: "Formatted at 2026-08-10 01:00:51 on dist-test-slave-gp6n"
server_key: "b046405b60ca7bb08f47c525aa53a411"
server_key_iv: "f0f2b62d7868fd4298e2d0cfad9246d4"
server_key_version: "kuduclusterkey@0"
I20260810 01:00:51.948582 2498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "506acb14a13c419e947d7d75455b49a7"
format_stamp: "Formatted at 2026-08-10 01:00:51 on dist-test-slave-gp6n"
server_key: "b046405b60ca7bb08f47c525aa53a411"
server_key_iv: "f0f2b62d7868fd4298e2d0cfad9246d4"
server_key_version: "kuduclusterkey@0"
I20260810 01:00:51.961478 2498 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.005s sys 0.008s
I20260810 01:00:51.999150 2520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:52.004632 2498 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.014s sys 0.002s
I20260810 01:00:52.004974 2498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "506acb14a13c419e947d7d75455b49a7"
format_stamp: "Formatted at 2026-08-10 01:00:51 on dist-test-slave-gp6n"
server_key: "b046405b60ca7bb08f47c525aa53a411"
server_key_iv: "f0f2b62d7868fd4298e2d0cfad9246d4"
server_key_version: "kuduclusterkey@0"
I20260810 01:00:52.005393 2498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:00:52.033238 2498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:00:52.047762 2498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:00:52.048226 2498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:00:52.058874 2498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:00:52.071422 2498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:00:52.071633 2498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:52.071851 2498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:00:52.071972 2498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 01:00:52 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323651, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
I20260810 01:00:52.409422 2498 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:34835
I20260810 01:00:52.409442 2647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:34835 every 8 connection(s)
I20260810 01:00:52.410933 2498 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 01:00:52.414054 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 2498
I20260810 01:00:52.414269 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 01:00:52.415359 20806 external_mini_cluster.cc:1428] Setting key 9a6c6a714ae0519aa56def0f80798e3b
Aug 10 01:00:52 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323651, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:00:52.446312 2651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:52.414730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:41879 (local address 127.20.81.129:55937)
0810 01:00:52.419038 (+ 4308us) negotiation.cc:107] Waiting for socket to connect
0810 01:00:52.419097 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 01:00:52.420876 (+ 1779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:00:52.422088 (+ 1212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:00:52.422101 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:00:52.422840 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:00:52.423883 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:52.423917 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:52.425561 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:52.425567 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:52.428161 (+ 2594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:52.428194 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:52.429452 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:52.429461 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:52.429856 (+ 395us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:52.434095 (+ 4239us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:00:52.434148 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:00:52.437189 (+ 3041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:00:52.441478 (+ 4289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:00:52.441487 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:00:52.441506 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:00:52.442126 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:00:52.442629 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:00:52.442633 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:00:52.442637 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:00:52.442911 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:00:52.443564 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:00:52.443573 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:00:52.444020 (+ 447us) client_negotiation.cc:769] Sending connection context
0810 01:00:52.444304 (+ 284us) client_negotiation.cc:240] Negotiation successful
0810 01:00:52.444838 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4003,"thread_start_us":3772,"threads_started":1}
I20260810 01:00:52.448437 2650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:52.413948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:55937 (local address 127.20.81.190:41879)
0810 01:00:52.421451 (+ 7503us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:52.421455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:52.421472 (+ 17us) server_negotiation.cc:407] Connection header received
0810 01:00:52.421520 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:52.421523 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:52.421584 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:52.421723 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:00:52.424132 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:52.425358 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:52.428373 (+ 3015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:52.429140 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:52.429305 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:52.437499 (+ 8194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:00:52.437506 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:00:52.437511 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:00:52.437553 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:00:52.441294 (+ 3741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:52.442299 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:52.442304 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:52.442307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:52.442491 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:52.443057 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:52.443063 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:52.443066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:52.443413 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:00:52.447640 (+ 4227us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:52.447785 (+ 145us) server_negotiation.cc:299] Negotiation successful
0810 01:00:52.448047 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7390,"thread_start_us":7280,"threads_started":1}
I20260810 01:00:52.450183 2648 heartbeater.cc:347] Connected to a master server at 127.20.81.190:41879
I20260810 01:00:52.450548 2648 heartbeater.cc:464] Registering TS with master...
I20260810 01:00:52.451458 2648 heartbeater.cc:511] Master 127.20.81.190:41879 requested a full tablet report, sending...
I20260810 01:00:52.454191 2402 ts_manager.cc:194] Registered new tserver with Master: 506acb14a13c419e947d7d75455b49a7 (127.20.81.129:34835)
I20260810 01:00:52.456349 2402 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:55937
WARNING: no policy specified for kudu/127.20.81.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
I20260810 01:00:52.506016 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.81.148:46607/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:41879
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 01:00:52 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323651, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
W20260810 01:00:52.997913 2655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:00:52.998262 2655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:00:52.998497 2655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:00:53.005932 2655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:00:53.006112 2655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:00:53.006304 2655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 01:00:53.017099 2655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.20.81.148:46607/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:41879
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:00:53.018605 2655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:00:53.020021 2655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:00:53.034653 2670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:00:53.034623 2672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:00:53.035583 2655 server_base.cc:1034] running on GCE node
W20260810 01:00:53.035845 2674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:00:53.036513 2655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:00:53.037578 2655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:00:53.038800 2655 hybrid_clock.cc:648] HybridClock initialized: now 1786323653038749 us; error 43 us; skew 500 ppm
I20260810 01:00:53.039177 2655 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:00:53 dist-test-slave-gp6n krb5kdc[1864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323653, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:00:53.042696 2655 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 01:00:53.044363 2655 webserver.cc:460] Webserver started at http://127.20.81.130:40107/ using document root <none> and password file <none>
I20260810 01:00:53.045056 2655 fs_manager.cc:359] Metadata directory not provided
I20260810 01:00:53.045185 2655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:00:53.045518 2655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 01:00:53 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323653, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
I20260810 01:00:53.064687 2655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "0a4aaa31a0eb4ffba30aa3402e3a0ad5"
format_stamp: "Formatted at 2026-08-10 01:00:53 on dist-test-slave-gp6n"
server_key: "2a361f3a0c97abfdc7ab05bfeb7dd167"
server_key_iv: "2b31c17fa42c3e5fcd6e9a2d05312216"
server_key_version: "kuduclusterkey@0"
I20260810 01:00:53.065824 2655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "0a4aaa31a0eb4ffba30aa3402e3a0ad5"
format_stamp: "Formatted at 2026-08-10 01:00:53 on dist-test-slave-gp6n"
server_key: "2a361f3a0c97abfdc7ab05bfeb7dd167"
server_key_iv: "2b31c17fa42c3e5fcd6e9a2d05312216"
server_key_version: "kuduclusterkey@0"
I20260810 01:00:53.072790 2655 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260810 01:00:53.089825 2683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:53.091571 2655 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:00:53.091779 2655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "0a4aaa31a0eb4ffba30aa3402e3a0ad5"
format_stamp: "Formatted at 2026-08-10 01:00:53 on dist-test-slave-gp6n"
server_key: "2a361f3a0c97abfdc7ab05bfeb7dd167"
server_key_iv: "2b31c17fa42c3e5fcd6e9a2d05312216"
server_key_version: "kuduclusterkey@0"
I20260810 01:00:53.092052 2655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:00:53.125123 2655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:00:53.129412 2655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:00:53.129740 2655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:00:53.130949 2655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:00:53.132479 2655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:00:53.132583 2655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:53.132717 2655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:00:53.132800 2655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:53.179345 2655 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:36723
I20260810 01:00:53.179395 2813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:36723 every 8 connection(s)
I20260810 01:00:53.181123 2655 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 01:00:53.191515 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 2655
I20260810 01:00:53.191710 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 01:00:53.192348 20806 external_mini_cluster.cc:1428] Setting key 001c351026bd81d7ed812f95c157fb4d
Aug 10 01:00:53 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323653, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:00:53.224871 2817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:53.185371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:54699 (local address 127.20.81.190:41879)
0810 01:00:53.198904 (+ 13533us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:53.198911 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:53.202461 (+ 3550us) server_negotiation.cc:407] Connection header received
0810 01:00:53.202533 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:53.202539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:53.202638 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:53.202731 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:00:53.204982 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:53.206130 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:53.210014 (+ 3884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:53.210674 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:53.210780 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:53.217017 (+ 6237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:00:53.217024 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:00:53.217029 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:00:53.217073 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:00:53.220773 (+ 3700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:53.221837 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:53.221843 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:53.221846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:53.221996 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:53.222551 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:53.222556 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:53.222560 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:53.222958 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:00:53.223083 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:53.224192 (+ 1109us) server_negotiation.cc:299] Negotiation successful
0810 01:00:53.224461 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13366,"thread_start_us":13256,"threads_started":1}
I20260810 01:00:53.227398 2816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:53.184726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:41879 (local address 127.20.81.130:54699)
0810 01:00:53.199492 (+ 14766us) negotiation.cc:107] Waiting for socket to connect
0810 01:00:53.199561 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0810 01:00:53.202045 (+ 2484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:00:53.203256 (+ 1211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:00:53.203268 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:00:53.203899 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:00:53.204761 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:53.204787 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:53.206332 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:53.206337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:53.209830 (+ 3493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:53.209855 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:53.211119 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:53.211126 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:53.211418 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:53.213493 (+ 2075us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:00:53.213525 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:00:53.216660 (+ 3135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:00:53.221001 (+ 4341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:00:53.221011 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:00:53.221030 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:00:53.221674 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:00:53.222147 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:00:53.222152 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:00:53.222155 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:00:53.222417 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:00:53.223197 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:00:53.223205 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:00:53.223689 (+ 484us) client_negotiation.cc:769] Sending connection context
0810 01:00:53.225194 (+ 1505us) client_negotiation.cc:240] Negotiation successful
0810 01:00:53.225781 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14438,"thread_start_us":24019,"threads_started":1}
I20260810 01:00:53.229241 2814 heartbeater.cc:347] Connected to a master server at 127.20.81.190:41879
I20260810 01:00:53.229609 2814 heartbeater.cc:464] Registering TS with master...
I20260810 01:00:53.230574 2814 heartbeater.cc:511] Master 127.20.81.190:41879 requested a full tablet report, sending...
I20260810 01:00:53.232424 2402 ts_manager.cc:194] Registered new tserver with Master: 0a4aaa31a0eb4ffba30aa3402e3a0ad5 (127.20.81.130:36723)
I20260810 01:00:53.233647 2402 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:54699
WARNING: no policy specified for kudu/127.20.81.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
I20260810 01:00:53.274863 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.81.148:46607/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:41879
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 01:00:53.460570 2648 heartbeater.cc:503] Master 127.20.81.190:41879 was elected leader, sending a full tablet report...
W20260810 01:00:53.632632 2820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:00:53.632964 2820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:00:53.633126 2820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:00:53.640632 2820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:00:53.640779 2820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:00:53.640950 2820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 01:00:53.651707 2820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.20.81.148:46607/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:41879
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:00:53.653095 2820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:00:53.654510 2820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:00:53.667870 2825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:00:53.667819 2826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:00:53.668198 2828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:00:53.669102 2820 server_base.cc:1034] running on GCE node
I20260810 01:00:53.669667 2820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:00:53.670604 2820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:00:53.671864 2820 hybrid_clock.cc:648] HybridClock initialized: now 1786323653671814 us; error 52 us; skew 500 ppm
I20260810 01:00:53.672226 2820 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:00:53 dist-test-slave-gp6n krb5kdc[1864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323653, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:00:53.675631 2820 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 01:00:53.677143 2820 webserver.cc:460] Webserver started at http://127.20.81.131:33847/ using document root <none> and password file <none>
I20260810 01:00:53.677868 2820 fs_manager.cc:359] Metadata directory not provided
I20260810 01:00:53.678025 2820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:00:53.678354 2820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 01:00:53 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323653, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
I20260810 01:00:53.695037 2820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "e53f380cb36040fea1ed78494f37b100"
format_stamp: "Formatted at 2026-08-10 01:00:53 on dist-test-slave-gp6n"
server_key: "85e55581f63ddd49aec06e8317b09fd9"
server_key_iv: "cf1feb55ea10760b308f4ae59be6270f"
server_key_version: "kuduclusterkey@0"
I20260810 01:00:53.696171 2820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "e53f380cb36040fea1ed78494f37b100"
format_stamp: "Formatted at 2026-08-10 01:00:53 on dist-test-slave-gp6n"
server_key: "85e55581f63ddd49aec06e8317b09fd9"
server_key_iv: "cf1feb55ea10760b308f4ae59be6270f"
server_key_version: "kuduclusterkey@0"
I20260810 01:00:53.703238 2820 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260810 01:00:53.719791 2837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:53.721455 2820 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:00:53.721633 2820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "e53f380cb36040fea1ed78494f37b100"
format_stamp: "Formatted at 2026-08-10 01:00:53 on dist-test-slave-gp6n"
server_key: "85e55581f63ddd49aec06e8317b09fd9"
server_key_iv: "cf1feb55ea10760b308f4ae59be6270f"
server_key_version: "kuduclusterkey@0"
I20260810 01:00:53.721874 2820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:00:53.754020 2820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:00:53.759590 2820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:00:53.759961 2820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:00:53.761346 2820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:00:53.763314 2820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:00:53.763442 2820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:53.763569 2820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:00:53.763643 2820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:53.797778 2820 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:42639
I20260810 01:00:53.797816 2968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:42639 every 8 connection(s)
I20260810 01:00:53.799227 2820 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 01:00:53.807217 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 2820
I20260810 01:00:53.807453 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 01:00:53.807935 20806 external_mini_cluster.cc:1428] Setting key afcf7fabdc17f76384ea44a93d9ab5f3
Aug 10 01:00:53 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323653, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:00:53.819991 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:53.801974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:53287 (local address 127.20.81.190:41879)
0810 01:00:53.802526 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:53.802531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:53.803463 (+ 932us) server_negotiation.cc:407] Connection header received
0810 01:00:53.805251 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:53.805257 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:53.805349 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:53.805472 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:00:53.806967 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:53.807816 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:53.809739 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:53.810307 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:53.810467 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:53.814000 (+ 3533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:00:53.814006 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:00:53.814009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:00:53.814040 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:00:53.816842 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:53.817609 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:53.817614 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:53.817618 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:53.817758 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:53.818191 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:53.818196 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:53.818198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:53.818530 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:00:53.818681 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:53.819365 (+ 684us) server_negotiation.cc:299] Negotiation successful
0810 01:00:53.819596 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":239,"threads_started":1}
I20260810 01:00:53.820950 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:53.802589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:41879 (local address 127.20.81.131:53287)
0810 01:00:53.803286 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0810 01:00:53.803334 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 01:00:53.804985 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:00:53.805718 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:00:53.805727 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:00:53.806135 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:00:53.806787 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:53.806805 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:53.807955 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:53.807961 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:53.809592 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:53.809604 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:53.810560 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:53.810568 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:53.810802 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:53.811563 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:00:53.811584 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:00:53.813776 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:00:53.817026 (+ 3250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:00:53.817034 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:00:53.817050 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:00:53.817482 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:00:53.817871 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:00:53.817876 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:00:53.817879 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:00:53.818063 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:00:53.818674 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:00:53.818681 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:00:53.819008 (+ 327us) client_negotiation.cc:769] Sending connection context
0810 01:00:53.819270 (+ 262us) client_negotiation.cc:240] Negotiation successful
0810 01:00:53.819804 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":289,"threads_started":1}
I20260810 01:00:53.822628 2969 heartbeater.cc:347] Connected to a master server at 127.20.81.190:41879
I20260810 01:00:53.822973 2969 heartbeater.cc:464] Registering TS with master...
I20260810 01:00:53.823704 2969 heartbeater.cc:511] Master 127.20.81.190:41879 requested a full tablet report, sending...
I20260810 01:00:53.825278 2402 ts_manager.cc:194] Registered new tserver with Master: e53f380cb36040fea1ed78494f37b100 (127.20.81.131:42639)
I20260810 01:00:53.826121 2402 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:53287
I20260810 01:00:53.836452 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 01:00:53.838210 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 2498
Aug 10 01:00:53 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.129@KRBTEST.COM
I20260810 01:00:53.855780 2976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:53.839236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43696 (local address 127.20.81.129:34835)
0810 01:00:53.839867 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:53.839874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:53.839903 (+ 29us) server_negotiation.cc:407] Connection header received
0810 01:00:53.840002 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:53.840010 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:53.840272 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:53.840427 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:00:53.842127 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:53.843303 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:53.844910 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:53.845540 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:53.845656 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:53.850018 (+ 4362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:00:53.850030 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:00:53.850045 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:00:53.850082 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:00:53.853010 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:53.853719 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:53.853726 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:53.853729 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:53.853877 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:53.854254 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:53.854259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:53.854261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:53.854729 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:00:53.854988 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:53.855247 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 01:00:53.855493 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":313,"threads_started":1}
I20260810 01:00:53.857072 2642 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:00:53.939980 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 2498
I20260810 01:00:53.960312 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 2655
Aug 10 01:00:53 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.130@KRBTEST.COM
I20260810 01:00:53.977566 2978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:53.961268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49992 (local address 127.20.81.130:36723)
0810 01:00:53.961905 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:53.961912 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:53.961938 (+ 26us) server_negotiation.cc:407] Connection header received
0810 01:00:53.962001 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:53.962008 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:53.962223 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:53.962357 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:00:53.963597 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:53.964587 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:53.966162 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:53.966899 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:53.967008 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:53.971494 (+ 4486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:00:53.971507 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:00:53.971522 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:00:53.971564 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:00:53.974501 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:53.975438 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:53.975446 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:53.975449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:53.975607 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:53.976038 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:53.976042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:53.976045 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:53.976437 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:00:53.976702 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:53.976997 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 01:00:53.977241 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":349,"threads_started":1}
I20260810 01:00:53.979074 2808 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:00:54.063504 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 2655
I20260810 01:00:54.087042 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 2820
Aug 10 01:00:54 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.131@KRBTEST.COM
I20260810 01:00:54.105515 2980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:54.088207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47528 (local address 127.20.81.131:42639)
0810 01:00:54.088904 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:54.088912 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:54.088943 (+ 31us) server_negotiation.cc:407] Connection header received
0810 01:00:54.089008 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:54.089015 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:54.089239 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:54.089431 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:00:54.090948 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:54.092135 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:54.093435 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:54.094080 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:54.094196 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:54.098921 (+ 4725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:00:54.098933 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:00:54.098949 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:00:54.098992 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:00:54.102593 (+ 3601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:54.103305 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:54.103311 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:54.103314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:54.103479 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:54.104019 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:54.104023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:54.104025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:54.104378 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:00:54.104645 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:54.104894 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 01:00:54.105098 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":331,"threads_started":1}
I20260810 01:00:54.106952 2964 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:00:54.192366 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 2820
I20260810 01:00:54.212385 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 2368
I20260810 01:00:54.213742 2438 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:00:54.327898 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 2368
I20260810 01:00:54.355155 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:41879
--webserver_interface=127.20.81.190
--webserver_port=40955
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:41879
--ranger_config_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.81.148:46607/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.81.148:46607/kms with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:00:54.713133 2983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:00:54.713629 2983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:00:54.713792 2983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:00:54.721765 2983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:00:54.721891 2983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:00:54.721997 2983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:00:54.722088 2983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:00:54.722168 2983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:00:54.733067 2983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.81.148:46607/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:41879
--ranger_config_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:41879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=40955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:00:54.734490 2983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:00:54.735934 2983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:00:54.748229 2989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:00:54.748303 2988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:00:54.748289 2991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:00:54.749541 2983 server_base.cc:1034] running on GCE node
I20260810 01:00:54.750319 2983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:00:54.751899 2983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:00:54.753127 2983 hybrid_clock.cc:648] HybridClock initialized: now 1786323654753067 us; error 58 us; skew 500 ppm
I20260810 01:00:54.753515 2983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:00:54 dist-test-slave-gp6n krb5kdc[1864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323654, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:00:54.756803 2983 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 01:00:54.758324 2983 webserver.cc:460] Webserver started at http://127.20.81.190:40955/ using document root <none> and password file <none>
I20260810 01:00:54.759037 2983 fs_manager.cc:359] Metadata directory not provided
I20260810 01:00:54.759158 2983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 01:00:54 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323654, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
I20260810 01:00:54.811811 2983 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 01:00:54.815285 2999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:54.816877 2983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 01:00:54.817032 2983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "4f78b9460c6043719a6f102ed85550c5"
format_stamp: "Formatted at 2026-08-10 01:00:50 on dist-test-slave-gp6n"
server_key: "1e22d91aacd0bf4c7e94032130587ed8"
server_key_iv: "f1f128257d96ce49681324c85531e820"
server_key_version: "kuduclusterkey@0"
I20260810 01:00:54.818008 2983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:00:54.839247 2983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:00:54.843413 2983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:00:54.843740 2983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:00:54.863176 2983 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:41879
I20260810 01:00:54.863214 3056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:41879 every 8 connection(s)
I20260810 01:00:54.864728 2983 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:00:54.871724 3057 sys_catalog.cc:263] Verifying existing consensus state
I20260810 01:00:54.874717 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 2983
I20260810 01:00:54.875187 3057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5: Bootstrap starting.
I20260810 01:00:54.875846 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.81.148:46607/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:34835
--local_ip_for_outbound_sockets=127.20.81.129
--tserver_master_addrs=127.20.81.190:41879
--webserver_port=36357
--webserver_interface=127.20.81.129
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.81.148:46607/kms with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 01:00:54.893864 3057 log.cc:824] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5: Log is configured to *not* fsync() on all Append() calls
I20260810 01:00:54.902798 3057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 01:00:54.903359 3057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5: Bootstrap complete.
I20260810 01:00:54.912425 3057 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f78b9460c6043719a6f102ed85550c5" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 41879 } }
I20260810 01:00:54.913425 3057 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f78b9460c6043719a6f102ed85550c5, State: Initialized, Role: FOLLOWER
I20260810 01:00:54.914036 3057 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f78b9460c6043719a6f102ed85550c5" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 41879 } }
I20260810 01:00:54.914279 3057 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:00:54.914407 3057 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:00:54.914636 3057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [term 1 FOLLOWER]: Advancing to term 2
I20260810 01:00:54.918558 3057 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f78b9460c6043719a6f102ed85550c5" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 41879 } }
I20260810 01:00:54.919039 3057 leader_election.cc:302] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f78b9460c6043719a6f102ed85550c5; no voters:
I20260810 01:00:54.919667 3057 leader_election.cc:288] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 01:00:54.919803 3061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 01:00:54.921001 3061 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [term 2 LEADER]: Becoming Leader. State: Replica: 4f78b9460c6043719a6f102ed85550c5, State: Running, Role: LEADER
I20260810 01:00:54.921553 3061 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f78b9460c6043719a6f102ed85550c5" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 41879 } }
I20260810 01:00:54.921922 3057 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:00:54.924368 3063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f78b9460c6043719a6f102ed85550c5. Latest consensus state: current_term: 2 leader_uuid: "4f78b9460c6043719a6f102ed85550c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f78b9460c6043719a6f102ed85550c5" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 41879 } } }
I20260810 01:00:54.924427 3062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4f78b9460c6043719a6f102ed85550c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f78b9460c6043719a6f102ed85550c5" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 41879 } } }
I20260810 01:00:54.924892 3062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [sys.catalog]: This master's current role is: LEADER
I20260810 01:00:54.927048 3063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5 [sys.catalog]: This master's current role is: LEADER
I20260810 01:00:54.939543 3070 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:00:54.945593 3070 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:00:54.946448 3057 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 01:00:54.947137 3070 catalog_manager.cc:1259] Loaded cluster ID: 91604ba4e19c4b6bb46e9301744189fc
I20260810 01:00:54.947235 3070 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:00:54.951105 3070 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:00:54.952484 3070 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5: Loaded TSK: 0
I20260810 01:00:54.954061 3070 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 01:00:55.329107 3059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:00:55.329551 3059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:00:55.329713 3059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:00:55.341802 3059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:00:55.341957 3059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:00:55.342149 3059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 01:00:55.358207 3059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.81.148:46607/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:34835
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=36357
--tserver_master_addrs=127.20.81.190:41879
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:00:55.359969 3059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:00:55.361685 3059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:00:55.384480 3097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:00:55.384647 3098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:00:55.392174 3101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:00:55.400364 3059 server_base.cc:1034] running on GCE node
I20260810 01:00:55.401479 3059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:00:55.402921 3059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:00:55.404191 3059 hybrid_clock.cc:648] HybridClock initialized: now 1786323655404123 us; error 57 us; skew 500 ppm
I20260810 01:00:55.404616 3059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:00:55 dist-test-slave-gp6n krb5kdc[1864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323655, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:00:55.412739 3059 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 01:00:55.416580 3059 webserver.cc:460] Webserver started at http://127.20.81.129:36357/ using document root <none> and password file <none>
I20260810 01:00:55.417630 3059 fs_manager.cc:359] Metadata directory not provided
I20260810 01:00:55.417824 3059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 01:00:55 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323655, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
I20260810 01:00:55.471556 3059 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.005s sys 0.000s
I20260810 01:00:55.485653 3110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:55.487883 3059 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.007s sys 0.009s
I20260810 01:00:55.488126 3059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "506acb14a13c419e947d7d75455b49a7"
format_stamp: "Formatted at 2026-08-10 01:00:51 on dist-test-slave-gp6n"
server_key: "b046405b60ca7bb08f47c525aa53a411"
server_key_iv: "f0f2b62d7868fd4298e2d0cfad9246d4"
server_key_version: "kuduclusterkey@0"
I20260810 01:00:55.489542 3059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:00:55.507083 3059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:00:55.512439 3059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:00:55.512779 3059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:00:55.515465 3059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:00:55.519245 3059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:00:55.519373 3059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:55.519503 3059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:00:55.519589 3059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:55.593932 3059 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:34835
I20260810 01:00:55.594026 3240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:34835 every 8 connection(s)
I20260810 01:00:55.595961 3059 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 01:00:55.605237 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 3059
I20260810 01:00:55.606102 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.81.148:46607/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:36723
--local_ip_for_outbound_sockets=127.20.81.130
--tserver_master_addrs=127.20.81.190:41879
--webserver_port=40107
--webserver_interface=127.20.81.130
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.81.148:46607/kms with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 01:00:55 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323655, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:00:55.627866 3243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:55.599951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:41879 (local address 127.20.81.129:55877)
0810 01:00:55.600791 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0810 01:00:55.600839 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 01:00:55.603288 (+ 2449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:00:55.605433 (+ 2145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:00:55.605445 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:00:55.606091 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:00:55.606871 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:55.606896 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:55.609394 (+ 2498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:55.609403 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:55.611611 (+ 2208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:55.611643 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:55.613210 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:55.613220 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:55.613596 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:55.614453 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:00:55.614492 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:00:55.617721 (+ 3229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:00:55.622998 (+ 5277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:00:55.623009 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:00:55.623025 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:00:55.623666 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:00:55.624131 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:00:55.624137 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:00:55.624141 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:00:55.624328 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:00:55.625131 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:00:55.625141 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:00:55.625501 (+ 360us) client_negotiation.cc:769] Sending connection context
0810 01:00:55.625792 (+ 291us) client_negotiation.cc:240] Negotiation successful
0810 01:00:55.626385 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":347,"threads_started":1}
I20260810 01:00:55.627866 3244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:55.600137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:55877 (local address 127.20.81.190:41879)
0810 01:00:55.603969 (+ 3832us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:55.603984 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:55.604021 (+ 37us) server_negotiation.cc:407] Connection header received
0810 01:00:55.604355 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:55.604368 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:55.604799 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:55.605256 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:00:55.607120 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:55.609176 (+ 2056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:55.611805 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:55.612952 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:55.614111 (+ 1159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:55.618046 (+ 3935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:00:55.618219 (+ 173us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:00:55.618250 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:00:55.618338 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:00:55.622767 (+ 4429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:55.623822 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:55.623830 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:55.623838 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:55.624009 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:55.624457 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:55.624462 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:55.624465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:55.624974 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:00:55.625206 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:55.626031 (+ 825us) server_negotiation.cc:299] Negotiation successful
0810 01:00:55.626429 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3600,"thread_start_us":3395,"threads_started":1}
I20260810 01:00:55.632596 3241 heartbeater.cc:347] Connected to a master server at 127.20.81.190:41879
I20260810 01:00:55.632939 3241 heartbeater.cc:464] Registering TS with master...
I20260810 01:00:55.633690 3241 heartbeater.cc:511] Master 127.20.81.190:41879 requested a full tablet report, sending...
W20260810 01:00:55.636716 3241 heartbeater.cc:644] Failed to heartbeat to 127.20.81.190:41879 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 01:00:55.636951 3241 heartbeater.cc:464] Registering TS with master...
I20260810 01:00:55.637162 3241 heartbeater.cc:511] Master 127.20.81.190:41879 requested a full tablet report, sending...
I20260810 01:00:55.638468 3241 heartbeater.cc:464] Registering TS with master...
I20260810 01:00:55.638754 3241 heartbeater.cc:511] Master 127.20.81.190:41879 requested a full tablet report, sending...
W20260810 01:00:55.639761 3241 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 01:00:56.078091 3245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:00:56.078483 3245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:00:56.078666 3245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:00:56.088254 3245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:00:56.088410 3245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:00:56.088611 3245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 01:00:56.099261 3245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.81.148:46607/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:36723
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=40107
--tserver_master_addrs=127.20.81.190:41879
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:00:56.100696 3245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:00:56.102053 3245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:00:56.117095 3264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:00:56.117723 3267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:00:56.118362 3262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:00:56.118659 3245 server_base.cc:1034] running on GCE node
I20260810 01:00:56.119424 3245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:00:56.120654 3245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:00:56.121891 3245 hybrid_clock.cc:648] HybridClock initialized: now 1786323656121826 us; error 43 us; skew 500 ppm
I20260810 01:00:56.122305 3245 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:00:56 dist-test-slave-gp6n krb5kdc[1864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323656, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:00:56.126607 3245 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 01:00:56.157774 3245 webserver.cc:460] Webserver started at http://127.20.81.130:40107/ using document root <none> and password file <none>
I20260810 01:00:56.158952 3245 fs_manager.cc:359] Metadata directory not provided
I20260810 01:00:56.159142 3245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 01:00:56 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323656, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
I20260810 01:00:56.207260 3245 fs_manager.cc:709] Time spent opening directory manager: real 0.017s user 0.002s sys 0.012s
I20260810 01:00:56.210945 3277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:56.212507 3245 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 01:00:56.212708 3245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "0a4aaa31a0eb4ffba30aa3402e3a0ad5"
format_stamp: "Formatted at 2026-08-10 01:00:53 on dist-test-slave-gp6n"
server_key: "2a361f3a0c97abfdc7ab05bfeb7dd167"
server_key_iv: "2b31c17fa42c3e5fcd6e9a2d05312216"
server_key_version: "kuduclusterkey@0"
I20260810 01:00:56.213802 3245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:00:56.244773 3245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:00:56.249305 3245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:00:56.249575 3245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:00:56.250720 3245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:00:56.252239 3245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:00:56.252338 3245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:56.252425 3245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:00:56.252481 3245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 01:00:56 dist-test-slave-gp6n krb5kdc[1864](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786323656, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:00:56.313344 3245 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:36723
I20260810 01:00:56.313381 3409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:36723 every 8 connection(s)
I20260810 01:00:56.314939 3245 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 01:00:56.323603 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 3245
I20260810 01:00:56.324381 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.81.148:46607/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:42639
--local_ip_for_outbound_sockets=127.20.81.131
--tserver_master_addrs=127.20.81.190:41879
--webserver_port=33847
--webserver_interface=127.20.81.131
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.81.148:46607/kms with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 01:00:56 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323656, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:00:56.348806 3416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:56.318440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:41879 (local address 127.20.81.130:38399)
0810 01:00:56.319468 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0810 01:00:56.319530 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 01:00:56.321990 (+ 2460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:00:56.322906 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:00:56.322918 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:00:56.323568 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:00:56.324459 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:56.324488 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:56.326069 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:56.326074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:56.328158 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:56.328175 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:56.329381 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:56.329390 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:56.329687 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:56.330586 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:00:56.330618 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:00:56.338181 (+ 7563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:00:56.342928 (+ 4747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:00:56.342941 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:00:56.342963 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:00:56.343651 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:00:56.344120 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:00:56.344128 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:00:56.344134 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:00:56.344387 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:00:56.345244 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:00:56.345255 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:00:56.345745 (+ 490us) client_negotiation.cc:769] Sending connection context
0810 01:00:56.346558 (+ 813us) client_negotiation.cc:240] Negotiation successful
0810 01:00:56.347202 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":334,"threads_started":1}
I20260810 01:00:56.350418 3415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:56.317884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:38399 (local address 127.20.81.190:41879)
0810 01:00:56.318464 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:56.318471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:56.319689 (+ 1218us) server_negotiation.cc:407] Connection header received
0810 01:00:56.322340 (+ 2651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:56.322344 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:56.322487 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:56.322590 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:00:56.324685 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:56.325869 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:56.328329 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:56.329079 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:56.329241 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:56.338509 (+ 9268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:00:56.338519 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:00:56.338526 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:00:56.338579 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:00:56.342705 (+ 4126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:56.343807 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:56.343815 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:56.343821 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:56.343988 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:56.344527 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:56.344534 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:56.344540 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:56.344974 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:00:56.345153 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:56.349612 (+ 4459us) server_negotiation.cc:299] Negotiation successful
0810 01:00:56.349879 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":280,"threads_started":1}
I20260810 01:00:56.351847 3411 heartbeater.cc:347] Connected to a master server at 127.20.81.190:41879
I20260810 01:00:56.352185 3411 heartbeater.cc:464] Registering TS with master...
I20260810 01:00:56.353155 3411 heartbeater.cc:511] Master 127.20.81.190:41879 requested a full tablet report, sending...
I20260810 01:00:56.355469 3017 ts_manager.cc:194] Registered new tserver with Master: 0a4aaa31a0eb4ffba30aa3402e3a0ad5 (127.20.81.130:36723)
I20260810 01:00:56.358067 3017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:38399
I20260810 01:00:56.639935 3241 heartbeater.cc:464] Registering TS with master...
I20260810 01:00:56.640412 3241 heartbeater.cc:511] Master 127.20.81.190:41879 requested a full tablet report, sending...
I20260810 01:00:56.641793 3017 ts_manager.cc:194] Registered new tserver with Master: 506acb14a13c419e947d7d75455b49a7 (127.20.81.129:34835)
I20260810 01:00:56.643019 3017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:55877
Aug 10 01:00:56 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323656, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
W20260810 01:00:56.827476 3418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:00:56.827931 3418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:00:56.828110 3418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:00:56.838311 3418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:00:56.838475 3418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:00:56.838649 3418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 01:00:56.853541 3418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.81.148:46607/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:42639
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=33847
--tserver_master_addrs=127.20.81.190:41879
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:00:56.855309 3418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:00:56.856999 3418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:00:56.886919 3423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:00:56.888620 3424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:00:56.896636 3426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:00:56.897166 3418 server_base.cc:1034] running on GCE node
I20260810 01:00:56.897960 3418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:00:56.901640 3418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:00:56.902872 3418 hybrid_clock.cc:648] HybridClock initialized: now 1786323656902799 us; error 60 us; skew 500 ppm
I20260810 01:00:56.903211 3418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:00:56 dist-test-slave-gp6n krb5kdc[1864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323656, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:00:56.906582 3418 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 01:00:56.911755 3418 webserver.cc:460] Webserver started at http://127.20.81.131:33847/ using document root <none> and password file <none>
I20260810 01:00:56.912566 3418 fs_manager.cc:359] Metadata directory not provided
I20260810 01:00:56.912732 3418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 01:00:56 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323656, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
I20260810 01:00:56.936290 3418 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 01:00:56.939534 3434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:56.941094 3418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:00:56.941283 3418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "e53f380cb36040fea1ed78494f37b100"
format_stamp: "Formatted at 2026-08-10 01:00:53 on dist-test-slave-gp6n"
server_key: "85e55581f63ddd49aec06e8317b09fd9"
server_key_iv: "cf1feb55ea10760b308f4ae59be6270f"
server_key_version: "kuduclusterkey@0"
I20260810 01:00:56.942306 3418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:00:56.965770 3418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:00:56.970582 3418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:00:56.970892 3418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:00:56.972229 3418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:00:56.973762 3418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:00:56.973896 3418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:00:56.974000 3418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:00:56.974068 3418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 01:00:57 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323656, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for HTTP/127.20.81.148@KRBTEST.COM
I20260810 01:00:57.117633 3558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:42639 every 8 connection(s)
I20260810 01:00:57.117985 3418 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:42639
I20260810 01:00:57.120388 3418 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 01:00:57.125051 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 3418
I20260810 01:00:57.186924 3561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:57.126718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58720 (local address 127.20.81.190:41879)
0810 01:00:57.153837 (+ 27119us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:57.153845 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:57.156022 (+ 2177us) server_negotiation.cc:407] Connection header received
0810 01:00:57.156116 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:57.156123 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:57.156209 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:57.156317 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:00:57.161814 (+ 5497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.163020 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.168909 (+ 5889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.169893 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.170082 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:57.173905 (+ 3823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:00:57.173911 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:00:57.173916 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:00:57.173971 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:00:57.177612 (+ 3641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:57.183605 (+ 5993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:57.183615 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:57.183621 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:57.183852 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:57.184849 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:57.184855 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:57.184858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:57.185273 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:00:57.185564 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:57.186021 (+ 457us) server_negotiation.cc:299] Negotiation successful
0810 01:00:57.186290 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26963,"thread_start_us":27138,"threads_started":1}
Aug 10 01:00:57 dist-test-slave-gp6n krb5kdc[1864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323656, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:00:57.207397 3563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:57.165134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:41879 (local address 127.20.81.131:47037)
0810 01:00:57.178580 (+ 13446us) negotiation.cc:107] Waiting for socket to connect
0810 01:00:57.178662 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0810 01:00:57.182314 (+ 3652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:00:57.186328 (+ 4014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:00:57.186340 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:00:57.187091 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:00:57.188137 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:57.188166 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.189612 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.189618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:57.192128 (+ 2510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:57.192145 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.193439 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.193447 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:57.193721 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:57.194668 (+ 947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:00:57.194699 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:00:57.197684 (+ 2985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:00:57.202389 (+ 4705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:00:57.202434 (+ 45us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:00:57.202454 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:00:57.203079 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:00:57.203545 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:00:57.203550 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:00:57.203554 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:00:57.203788 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:00:57.204525 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:00:57.204533 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:00:57.204968 (+ 435us) client_negotiation.cc:769] Sending connection context
0810 01:00:57.205277 (+ 309us) client_negotiation.cc:240] Negotiation successful
0810 01:00:57.205857 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12998,"thread_start_us":17135,"threads_started":1}
I20260810 01:00:57.208534 3564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:57.165748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:47037 (local address 127.20.81.190:41879)
0810 01:00:57.184808 (+ 19060us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:57.184815 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:57.184833 (+ 18us) server_negotiation.cc:407] Connection header received
0810 01:00:57.185568 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:57.185575 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:57.185655 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:57.185756 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:00:57.188396 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.189453 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.192322 (+ 2869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.192997 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.193123 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:57.198005 (+ 4882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:00:57.198014 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:00:57.198019 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:00:57.198066 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:00:57.202179 (+ 4113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:57.203230 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:57.203235 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:57.203239 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:57.203404 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:57.203936 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:57.203941 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:57.203945 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:57.204286 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:00:57.204449 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:57.207921 (+ 3472us) server_negotiation.cc:299] Negotiation successful
0810 01:00:57.208143 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18906,"thread_start_us":19080,"threads_started":1}
I20260810 01:00:57.210572 3559 heartbeater.cc:347] Connected to a master server at 127.20.81.190:41879
I20260810 01:00:57.210917 3559 heartbeater.cc:464] Registering TS with master...
I20260810 01:00:57.211800 3559 heartbeater.cc:511] Master 127.20.81.190:41879 requested a full tablet report, sending...
I20260810 01:00:57.213873 3017 ts_manager.cc:194] Registered new tserver with Master: e53f380cb36040fea1ed78494f37b100 (127.20.81.131:42639)
I20260810 01:00:57.215046 3017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:47037
I20260810 01:00:57.217548 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 01:00:57.245823 3564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:57.221906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58732 (local address 127.20.81.190:41879)
0810 01:00:57.222084 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:57.222090 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:57.227285 (+ 5195us) server_negotiation.cc:407] Connection header received
0810 01:00:57.227612 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:57.227619 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:57.227708 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:57.227841 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:00:57.230140 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.231307 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.233578 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.234224 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.234340 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:57.238359 (+ 4019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:00:57.238373 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:00:57.238406 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:00:57.238480 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:00:57.242220 (+ 3740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:57.243246 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:57.243251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:57.243254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:57.243421 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:57.243908 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:57.243913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:57.243916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:57.244312 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:00:57.244558 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:57.245195 (+ 637us) server_negotiation.cc:299] Negotiation successful
0810 01:00:57.245457 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 01:00:57.253275 3017 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 01:00:57.264945 3017 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 01:00:57.331058 3574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:57.284232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:34835 (local address 127.0.0.1:43700)
0810 01:00:57.296341 (+ 12109us) negotiation.cc:107] Waiting for socket to connect
0810 01:00:57.296362 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 01:00:57.296480 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:00:57.315581 (+ 19101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:00:57.315589 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:00:57.315627 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:00:57.316054 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:57.316068 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.318426 (+ 2358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.318432 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:57.322085 (+ 3653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:57.322102 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.329994 (+ 7892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.330001 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:57.330238 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:57.330301 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 01:00:57.330491 (+ 190us) client_negotiation.cc:240] Negotiation successful
0810 01:00:57.330692 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11980,"thread_start_us":9917,"threads_started":1}
I20260810 01:00:57.331926 3578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:57.285315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43700 (local address 127.20.81.129:34835)
0810 01:00:57.314848 (+ 29533us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:57.314859 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:57.314893 (+ 34us) server_negotiation.cc:407] Connection header received
0810 01:00:57.315000 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:57.315008 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:57.315301 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:57.315553 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:00:57.316295 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.317993 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.322341 (+ 4348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.329380 (+ 7039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.329522 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:57.331275 (+ 1753us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:57.331374 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 01:00:57.331565 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29317,"thread_start_us":30407,"threads_started":1}
I20260810 01:00:57.333737 3576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:57.283682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:42639 (local address 127.0.0.1:47536)
0810 01:00:57.293675 (+ 9993us) negotiation.cc:107] Waiting for socket to connect
0810 01:00:57.293727 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 01:00:57.293944 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:00:57.324015 (+ 30071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:00:57.324021 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:00:57.324051 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:00:57.324524 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:57.324536 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.326729 (+ 2193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.326735 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:57.328975 (+ 2240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:57.328992 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.333006 (+ 4014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.333014 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:57.333190 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:57.333223 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 01:00:57.333291 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 01:00:57.333423 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9691,"thread_start_us":12396,"threads_started":1}
I20260810 01:00:57.335137 3577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:57.284668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47536 (local address 127.20.81.131:42639)
0810 01:00:57.323151 (+ 38483us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:57.323163 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:57.323211 (+ 48us) server_negotiation.cc:407] Connection header received
0810 01:00:57.323424 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:57.323436 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:57.323769 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:57.324031 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:00:57.324774 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.326556 (+ 1782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.329196 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.332743 (+ 3547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.334283 (+ 1540us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:57.334537 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:57.334618 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 01:00:57.334774 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37632,"thread_start_us":38062,"threads_started":1}
I20260810 01:00:57.337864 3164 tablet_service.cc:1467] Processing CreateTablet for tablet 4ac3b0bafa26410682968fb8ec76667d (DEFAULT_TABLE table=test-table [id=3f028180bd874f25abfd8bab099120e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:00:57.338917 3490 tablet_service.cc:1467] Processing CreateTablet for tablet 4ac3b0bafa26410682968fb8ec76667d (DEFAULT_TABLE table=test-table [id=3f028180bd874f25abfd8bab099120e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:00:57.339586 3164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ac3b0bafa26410682968fb8ec76667d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:00:57.340417 3490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ac3b0bafa26410682968fb8ec76667d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:00:57.351361 3579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:57.296779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:36723 (local address 127.0.0.1:50004)
0810 01:00:57.324868 (+ 28089us) negotiation.cc:107] Waiting for socket to connect
0810 01:00:57.324886 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 01:00:57.324973 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:00:57.328414 (+ 3441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:00:57.328421 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:00:57.328455 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:00:57.328871 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:57.328885 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.345344 (+ 16459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.345354 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:57.347754 (+ 2400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:57.347774 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.350479 (+ 2705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.350487 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:57.350686 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:57.350722 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 01:00:57.350823 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 01:00:57.350972 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27964,"thread_start_us":27799,"threads_started":1}
I20260810 01:00:57.355227 3580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:57.296828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50004 (local address 127.20.81.130:36723)
0810 01:00:57.327737 (+ 30909us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:57.327746 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:57.327775 (+ 29us) server_negotiation.cc:407] Connection header received
0810 01:00:57.327872 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:57.327879 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:57.328108 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:57.328336 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:00:57.331564 (+ 3228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.344360 (+ 12796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.348139 (+ 3779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.349858 (+ 1719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.350019 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:57.350271 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:57.354648 (+ 4377us) server_negotiation.cc:299] Negotiation successful
0810 01:00:57.354858 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30743,"thread_start_us":31716,"threads_started":1}
I20260810 01:00:57.359325 3332 tablet_service.cc:1467] Processing CreateTablet for tablet 4ac3b0bafa26410682968fb8ec76667d (DEFAULT_TABLE table=test-table [id=3f028180bd874f25abfd8bab099120e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:00:57.360930 3332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ac3b0bafa26410682968fb8ec76667d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:00:57.363705 3411 heartbeater.cc:503] Master 127.20.81.190:41879 was elected leader, sending a full tablet report...
I20260810 01:00:57.365736 3583 tablet_bootstrap.cc:492] T 4ac3b0bafa26410682968fb8ec76667d P 506acb14a13c419e947d7d75455b49a7: Bootstrap starting.
I20260810 01:00:57.370826 3584 tablet_bootstrap.cc:492] T 4ac3b0bafa26410682968fb8ec76667d P e53f380cb36040fea1ed78494f37b100: Bootstrap starting.
I20260810 01:00:57.374833 3583 tablet_bootstrap.cc:654] T 4ac3b0bafa26410682968fb8ec76667d P 506acb14a13c419e947d7d75455b49a7: Neither blocks nor log segments found. Creating new log.
I20260810 01:00:57.376025 3584 tablet_bootstrap.cc:654] T 4ac3b0bafa26410682968fb8ec76667d P e53f380cb36040fea1ed78494f37b100: Neither blocks nor log segments found. Creating new log.
I20260810 01:00:57.376248 3583 log.cc:824] T 4ac3b0bafa26410682968fb8ec76667d P 506acb14a13c419e947d7d75455b49a7: Log is configured to *not* fsync() on all Append() calls
I20260810 01:00:57.377382 3584 log.cc:824] T 4ac3b0bafa26410682968fb8ec76667d P e53f380cb36040fea1ed78494f37b100: Log is configured to *not* fsync() on all Append() calls
I20260810 01:00:57.384020 3584 tablet_bootstrap.cc:492] T 4ac3b0bafa26410682968fb8ec76667d P e53f380cb36040fea1ed78494f37b100: No bootstrap required, opened a new log
I20260810 01:00:57.384603 3584 ts_tablet_manager.cc:1397] T 4ac3b0bafa26410682968fb8ec76667d P e53f380cb36040fea1ed78494f37b100: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.003s
I20260810 01:00:57.386225 3590 tablet_bootstrap.cc:492] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5: Bootstrap starting.
I20260810 01:00:57.390877 3590 tablet_bootstrap.cc:654] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5: Neither blocks nor log segments found. Creating new log.
I20260810 01:00:57.391435 3583 tablet_bootstrap.cc:492] T 4ac3b0bafa26410682968fb8ec76667d P 506acb14a13c419e947d7d75455b49a7: No bootstrap required, opened a new log
I20260810 01:00:57.391774 3583 ts_tablet_manager.cc:1397] T 4ac3b0bafa26410682968fb8ec76667d P 506acb14a13c419e947d7d75455b49a7: Time spent bootstrapping tablet: real 0.028s user 0.012s sys 0.007s
I20260810 01:00:57.392241 3590 log.cc:824] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5: Log is configured to *not* fsync() on all Append() calls
I20260810 01:00:57.395670 3590 tablet_bootstrap.cc:492] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5: No bootstrap required, opened a new log
I20260810 01:00:57.395994 3590 ts_tablet_manager.cc:1397] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260810 01:00:57.395956 3584 raft_consensus.cc:348] T 4ac3b0bafa26410682968fb8ec76667d P e53f380cb36040fea1ed78494f37b100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e53f380cb36040fea1ed78494f37b100" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42639 } } peers { permanent_uuid: "0a4aaa31a0eb4ffba30aa3402e3a0ad5" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36723 } } peers { permanent_uuid: "506acb14a13c419e947d7d75455b49a7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 34835 } }
I20260810 01:00:57.396436 3584 raft_consensus.cc:374] T 4ac3b0bafa26410682968fb8ec76667d P e53f380cb36040fea1ed78494f37b100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:00:57.396540 3584 raft_consensus.cc:729] T 4ac3b0bafa26410682968fb8ec76667d P e53f380cb36040fea1ed78494f37b100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e53f380cb36040fea1ed78494f37b100, State: Initialized, Role: FOLLOWER
I20260810 01:00:57.397181 3584 consensus_queue.cc:260] T 4ac3b0bafa26410682968fb8ec76667d P e53f380cb36040fea1ed78494f37b100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e53f380cb36040fea1ed78494f37b100" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42639 } } peers { permanent_uuid: "0a4aaa31a0eb4ffba30aa3402e3a0ad5" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36723 } } peers { permanent_uuid: "506acb14a13c419e947d7d75455b49a7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 34835 } }
I20260810 01:00:57.398438 3559 heartbeater.cc:503] Master 127.20.81.190:41879 was elected leader, sending a full tablet report...
I20260810 01:00:57.399065 3584 ts_tablet_manager.cc:1428] T 4ac3b0bafa26410682968fb8ec76667d P e53f380cb36040fea1ed78494f37b100: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260810 01:00:57.402468 3583 raft_consensus.cc:348] T 4ac3b0bafa26410682968fb8ec76667d P 506acb14a13c419e947d7d75455b49a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e53f380cb36040fea1ed78494f37b100" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42639 } } peers { permanent_uuid: "0a4aaa31a0eb4ffba30aa3402e3a0ad5" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36723 } } peers { permanent_uuid: "506acb14a13c419e947d7d75455b49a7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 34835 } }
I20260810 01:00:57.403043 3583 raft_consensus.cc:374] T 4ac3b0bafa26410682968fb8ec76667d P 506acb14a13c419e947d7d75455b49a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:00:57.403215 3583 raft_consensus.cc:729] T 4ac3b0bafa26410682968fb8ec76667d P 506acb14a13c419e947d7d75455b49a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506acb14a13c419e947d7d75455b49a7, State: Initialized, Role: FOLLOWER
I20260810 01:00:57.403906 3583 consensus_queue.cc:260] T 4ac3b0bafa26410682968fb8ec76667d P 506acb14a13c419e947d7d75455b49a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e53f380cb36040fea1ed78494f37b100" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42639 } } peers { permanent_uuid: "0a4aaa31a0eb4ffba30aa3402e3a0ad5" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36723 } } peers { permanent_uuid: "506acb14a13c419e947d7d75455b49a7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 34835 } }
I20260810 01:00:57.405078 3241 heartbeater.cc:503] Master 127.20.81.190:41879 was elected leader, sending a full tablet report...
I20260810 01:00:57.405828 3583 ts_tablet_manager.cc:1428] T 4ac3b0bafa26410682968fb8ec76667d P 506acb14a13c419e947d7d75455b49a7: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260810 01:00:57.406179 3590 raft_consensus.cc:348] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e53f380cb36040fea1ed78494f37b100" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42639 } } peers { permanent_uuid: "0a4aaa31a0eb4ffba30aa3402e3a0ad5" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36723 } } peers { permanent_uuid: "506acb14a13c419e947d7d75455b49a7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 34835 } }
I20260810 01:00:57.406679 3590 raft_consensus.cc:374] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:00:57.406785 3590 raft_consensus.cc:729] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a4aaa31a0eb4ffba30aa3402e3a0ad5, State: Initialized, Role: FOLLOWER
I20260810 01:00:57.407435 3590 consensus_queue.cc:260] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e53f380cb36040fea1ed78494f37b100" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42639 } } peers { permanent_uuid: "0a4aaa31a0eb4ffba30aa3402e3a0ad5" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36723 } } peers { permanent_uuid: "506acb14a13c419e947d7d75455b49a7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 34835 } }
I20260810 01:00:57.409180 3590 ts_tablet_manager.cc:1428] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5: Time spent starting tablet: real 0.013s user 0.007s sys 0.004s
W20260810 01:00:57.436144 3562 tablet.cc:2369] T 4ac3b0bafa26410682968fb8ec76667d P e53f380cb36040fea1ed78494f37b100: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:00:57.455924 3598 raft_consensus.cc:482] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:00:57.456271 3598 raft_consensus.cc:504] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e53f380cb36040fea1ed78494f37b100" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42639 } } peers { permanent_uuid: "0a4aaa31a0eb4ffba30aa3402e3a0ad5" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36723 } } peers { permanent_uuid: "506acb14a13c419e947d7d75455b49a7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 34835 } }
I20260810 01:00:57.458180 3598 leader_election.cc:288] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e53f380cb36040fea1ed78494f37b100 (127.20.81.131:42639), 506acb14a13c419e947d7d75455b49a7 (127.20.81.129:34835)
I20260810 01:00:57.465242 3577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:57.458585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:59261 (local address 127.20.81.131:42639)
0810 01:00:57.458784 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:57.458791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:57.459680 (+ 889us) server_negotiation.cc:407] Connection header received
0810 01:00:57.459928 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:57.459938 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:57.459998 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:57.460101 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:00:57.460750 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.461472 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.463481 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.464235 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.464446 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:57.464524 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:57.464835 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 01:00:57.464962 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 01:00:57.465260 3578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:57.458779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:50077 (local address 127.20.81.129:34835)
0810 01:00:57.459055 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:57.459065 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:57.459872 (+ 807us) server_negotiation.cc:407] Connection header received
0810 01:00:57.460065 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:57.460070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:57.460132 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:57.460233 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:00:57.460891 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.461612 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.463492 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.464253 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.464453 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:57.464535 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:57.464881 (+ 346us) server_negotiation.cc:299] Negotiation successful
0810 01:00:57.465010 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 01:00:57.465428 3599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:57.458420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:42639 (local address 127.20.81.130:59261)
0810 01:00:57.459550 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0810 01:00:57.459571 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 01:00:57.459765 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:00:57.460117 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:00:57.460121 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:00:57.460145 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:00:57.460585 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:57.460594 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.461608 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.461616 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:57.463315 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:57.463327 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.464478 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.464484 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:57.464639 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:57.464697 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 01:00:57.464808 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 01:00:57.464945 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":904,"mutex_wait_us":132,"thread_start_us":640,"threads_started":1}
I20260810 01:00:57.465533 3601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:57.458707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:34835 (local address 127.20.81.130:50077)
0810 01:00:57.459759 (+ 1052us) negotiation.cc:107] Waiting for socket to connect
0810 01:00:57.459773 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 01:00:57.459931 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:00:57.460250 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:00:57.460254 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:00:57.460293 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:00:57.460718 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:57.460727 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.461764 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.461769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:57.463322 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:00:57.463334 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.464523 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.464529 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:00:57.464716 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:57.464739 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 01:00:57.464849 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 01:00:57.464968 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"mutex_wait_us":55,"thread_start_us":754,"threads_started":1}
I20260810 01:00:57.466533 3512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ac3b0bafa26410682968fb8ec76667d" candidate_uuid: "0a4aaa31a0eb4ffba30aa3402e3a0ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e53f380cb36040fea1ed78494f37b100" is_pre_election: true
I20260810 01:00:57.466609 3186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ac3b0bafa26410682968fb8ec76667d" candidate_uuid: "0a4aaa31a0eb4ffba30aa3402e3a0ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "506acb14a13c419e947d7d75455b49a7" is_pre_election: true
I20260810 01:00:57.467085 3512 raft_consensus.cc:2454] T 4ac3b0bafa26410682968fb8ec76667d P e53f380cb36040fea1ed78494f37b100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a4aaa31a0eb4ffba30aa3402e3a0ad5 in term 0.
I20260810 01:00:57.467128 3186 raft_consensus.cc:2454] T 4ac3b0bafa26410682968fb8ec76667d P 506acb14a13c419e947d7d75455b49a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a4aaa31a0eb4ffba30aa3402e3a0ad5 in term 0.
I20260810 01:00:57.467753 3281 leader_election.cc:302] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a4aaa31a0eb4ffba30aa3402e3a0ad5, e53f380cb36040fea1ed78494f37b100; no voters:
I20260810 01:00:57.468147 3598 raft_consensus.cc:2781] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:00:57.468297 3598 raft_consensus.cc:482] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:00:57.468407 3598 raft_consensus.cc:3037] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:00:57.470693 3598 raft_consensus.cc:504] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e53f380cb36040fea1ed78494f37b100" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42639 } } peers { permanent_uuid: "0a4aaa31a0eb4ffba30aa3402e3a0ad5" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36723 } } peers { permanent_uuid: "506acb14a13c419e947d7d75455b49a7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 34835 } }
I20260810 01:00:57.471323 3598 leader_election.cc:288] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5 [CANDIDATE]: Term 1 election: Requested vote from peers e53f380cb36040fea1ed78494f37b100 (127.20.81.131:42639), 506acb14a13c419e947d7d75455b49a7 (127.20.81.129:34835)
I20260810 01:00:57.471799 3512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ac3b0bafa26410682968fb8ec76667d" candidate_uuid: "0a4aaa31a0eb4ffba30aa3402e3a0ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e53f380cb36040fea1ed78494f37b100"
I20260810 01:00:57.471868 3186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ac3b0bafa26410682968fb8ec76667d" candidate_uuid: "0a4aaa31a0eb4ffba30aa3402e3a0ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "506acb14a13c419e947d7d75455b49a7"
I20260810 01:00:57.472024 3512 raft_consensus.cc:3037] T 4ac3b0bafa26410682968fb8ec76667d P e53f380cb36040fea1ed78494f37b100 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:00:57.472076 3186 raft_consensus.cc:3037] T 4ac3b0bafa26410682968fb8ec76667d P 506acb14a13c419e947d7d75455b49a7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:00:57.475374 3512 raft_consensus.cc:2454] T 4ac3b0bafa26410682968fb8ec76667d P e53f380cb36040fea1ed78494f37b100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a4aaa31a0eb4ffba30aa3402e3a0ad5 in term 1.
I20260810 01:00:57.475365 3186 raft_consensus.cc:2454] T 4ac3b0bafa26410682968fb8ec76667d P 506acb14a13c419e947d7d75455b49a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a4aaa31a0eb4ffba30aa3402e3a0ad5 in term 1.
I20260810 01:00:57.475826 3281 leader_election.cc:302] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0a4aaa31a0eb4ffba30aa3402e3a0ad5, 506acb14a13c419e947d7d75455b49a7, e53f380cb36040fea1ed78494f37b100; no voters:
I20260810 01:00:57.476141 3598 raft_consensus.cc:2781] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:00:57.476655 3598 raft_consensus.cc:686] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5 [term 1 LEADER]: Becoming Leader. State: Replica: 0a4aaa31a0eb4ffba30aa3402e3a0ad5, State: Running, Role: LEADER
I20260810 01:00:57.477128 3598 consensus_queue.cc:237] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e53f380cb36040fea1ed78494f37b100" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42639 } } peers { permanent_uuid: "0a4aaa31a0eb4ffba30aa3402e3a0ad5" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36723 } } peers { permanent_uuid: "506acb14a13c419e947d7d75455b49a7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 34835 } }
I20260810 01:00:57.482177 3017 catalog_manager.cc:5495] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a4aaa31a0eb4ffba30aa3402e3a0ad5 (127.20.81.130). New cstate: current_term: 1 leader_uuid: "0a4aaa31a0eb4ffba30aa3402e3a0ad5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e53f380cb36040fea1ed78494f37b100" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a4aaa31a0eb4ffba30aa3402e3a0ad5" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "506acb14a13c419e947d7d75455b49a7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260810 01:00:57.540405 3580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:57.534658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50018 (local address 127.20.81.130:36723)
0810 01:00:57.534893 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:57.534900 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:57.535010 (+ 110us) server_negotiation.cc:407] Connection header received
0810 01:00:57.535281 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:57.535287 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:57.535384 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:57.535550 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:00:57.535981 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.536756 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.538147 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.538636 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.538777 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:57.539301 (+ 524us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:00:57.539812 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:00:57.539988 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:57.540045 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 01:00:57.540125 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 01:00:57.548717 3186 raft_consensus.cc:1261] T 4ac3b0bafa26410682968fb8ec76667d P 506acb14a13c419e947d7d75455b49a7 [term 1 FOLLOWER]: Refusing update from remote peer 0a4aaa31a0eb4ffba30aa3402e3a0ad5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:00:57.548722 3512 raft_consensus.cc:1261] T 4ac3b0bafa26410682968fb8ec76667d P e53f380cb36040fea1ed78494f37b100 [term 1 FOLLOWER]: Refusing update from remote peer 0a4aaa31a0eb4ffba30aa3402e3a0ad5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:00:57.549782 3598 consensus_queue.cc:1035] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e53f380cb36040fea1ed78494f37b100" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:00:57.550109 3602 consensus_queue.cc:1035] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "506acb14a13c419e947d7d75455b49a7" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:00:57.563697 3607 mvcc.cc:205] Tried to move back new op lower bound from 7316781701309583360 to 7316781701030268928. Current Snapshot: MvccSnapshot[applied={T|T < 7316781701309583360}]
I20260810 01:00:57.565112 3608 mvcc.cc:205] Tried to move back new op lower bound from 7316781701309583360 to 7316781701030268928. Current Snapshot: MvccSnapshot[applied={T|T < 7316781701309583360}]
I20260810 01:00:57.582520 3017 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:00:57.582811 3017 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:00:57.586231 3017 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f78b9460c6043719a6f102ed85550c5: Sending DeleteTablet for 3 replicas of tablet 4ac3b0bafa26410682968fb8ec76667d
I20260810 01:00:57.587469 3490 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ac3b0bafa26410682968fb8ec76667d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:00:57 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:47536
I20260810 01:00:57.587507 3332 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ac3b0bafa26410682968fb8ec76667d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:00:57 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:50004
I20260810 01:00:57.587888 3164 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ac3b0bafa26410682968fb8ec76667d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:00:57 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:43700
I20260810 01:00:57.595988 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 3059
I20260810 01:00:57.600023 3620 tablet_replica.cc:330] T 4ac3b0bafa26410682968fb8ec76667d P 506acb14a13c419e947d7d75455b49a7: stopping tablet replica
I20260810 01:00:57.600539 3620 raft_consensus.cc:2229] T 4ac3b0bafa26410682968fb8ec76667d P 506acb14a13c419e947d7d75455b49a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:00:57.601121 3620 raft_consensus.cc:2258] T 4ac3b0bafa26410682968fb8ec76667d P 506acb14a13c419e947d7d75455b49a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:00:57.605986 3620 ts_tablet_manager.cc:1905] T 4ac3b0bafa26410682968fb8ec76667d P 506acb14a13c419e947d7d75455b49a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:00:57.612007 3618 tablet_replica.cc:330] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5: stopping tablet replica
I20260810 01:00:57.612603 3618 raft_consensus.cc:2229] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5 [term 1 LEADER]: Raft consensus shutting down.
I20260810 01:00:57.613364 3618 raft_consensus.cc:2258] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:00:57.618700 3620 ts_tablet_manager.cc:1918] T 4ac3b0bafa26410682968fb8ec76667d P 506acb14a13c419e947d7d75455b49a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:00:57.618963 3620 log.cc:1196] T 4ac3b0bafa26410682968fb8ec76667d P 506acb14a13c419e947d7d75455b49a7: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-0/wal/wals/4ac3b0bafa26410682968fb8ec76667d
I20260810 01:00:57.619773 3620 ts_tablet_manager.cc:1939] T 4ac3b0bafa26410682968fb8ec76667d P 506acb14a13c419e947d7d75455b49a7: Deleting consensus metadata
I20260810 01:00:57.622174 3001 catalog_manager.cc:4841] TS 506acb14a13c419e947d7d75455b49a7 (127.20.81.129:34835): tablet 4ac3b0bafa26410682968fb8ec76667d (table test-table [id=3f028180bd874f25abfd8bab099120e7]) successfully deleted
I20260810 01:00:57.622704 3578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:57.598579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43704 (local address 127.20.81.129:34835)
0810 01:00:57.598766 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:57.598772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:57.598793 (+ 21us) server_negotiation.cc:407] Connection header received
0810 01:00:57.598867 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:57.598872 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:57.598962 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:57.599089 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:00:57.601221 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.602450 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.604930 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.605618 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.605747 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:57.614793 (+ 9046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:00:57.614807 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:00:57.614825 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:00:57.614878 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:00:57.619054 (+ 4176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:57.620077 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:57.620087 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:57.620091 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:57.620273 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:57.620685 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:57.620690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:57.620693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:57.621172 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:00:57.621570 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:57.621879 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 01:00:57.622178 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 01:00:57.623708 3236 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:00:57.628104 3618 ts_tablet_manager.cc:1905] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:00:57.628456 3617 tablet_replica.cc:330] T 4ac3b0bafa26410682968fb8ec76667d P e53f380cb36040fea1ed78494f37b100: stopping tablet replica
I20260810 01:00:57.628949 3617 raft_consensus.cc:2229] T 4ac3b0bafa26410682968fb8ec76667d P e53f380cb36040fea1ed78494f37b100 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:00:57.629703 3617 raft_consensus.cc:2258] T 4ac3b0bafa26410682968fb8ec76667d P e53f380cb36040fea1ed78494f37b100 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:00:57.632567 3617 ts_tablet_manager.cc:1905] T 4ac3b0bafa26410682968fb8ec76667d P e53f380cb36040fea1ed78494f37b100: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:00:57.637869 3618 ts_tablet_manager.cc:1918] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:00:57.638160 3618 log.cc:1196] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/4ac3b0bafa26410682968fb8ec76667d
I20260810 01:00:57.638895 3618 ts_tablet_manager.cc:1939] T 4ac3b0bafa26410682968fb8ec76667d P 0a4aaa31a0eb4ffba30aa3402e3a0ad5: Deleting consensus metadata
I20260810 01:00:57.639647 3617 ts_tablet_manager.cc:1918] T 4ac3b0bafa26410682968fb8ec76667d P e53f380cb36040fea1ed78494f37b100: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:00:57.639858 3617 log.cc:1196] T 4ac3b0bafa26410682968fb8ec76667d P e53f380cb36040fea1ed78494f37b100: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/4ac3b0bafa26410682968fb8ec76667d
I20260810 01:00:57.640405 3617 ts_tablet_manager.cc:1939] T 4ac3b0bafa26410682968fb8ec76667d P e53f380cb36040fea1ed78494f37b100: Deleting consensus metadata
I20260810 01:00:57.641309 3003 catalog_manager.cc:4841] TS 0a4aaa31a0eb4ffba30aa3402e3a0ad5 (127.20.81.130:36723): tablet 4ac3b0bafa26410682968fb8ec76667d (table test-table [id=3f028180bd874f25abfd8bab099120e7]) successfully deleted
I20260810 01:00:57.642159 3003 catalog_manager.cc:4841] TS e53f380cb36040fea1ed78494f37b100 (127.20.81.131:42639): tablet 4ac3b0bafa26410682968fb8ec76667d (table test-table [id=3f028180bd874f25abfd8bab099120e7]) successfully deleted
I20260810 01:00:57.721061 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 3059
I20260810 01:00:57.743552 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 3245
I20260810 01:00:57.760564 3580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:57.744462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50032 (local address 127.20.81.130:36723)
0810 01:00:57.744677 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:57.744683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:57.744705 (+ 22us) server_negotiation.cc:407] Connection header received
0810 01:00:57.744863 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:57.744869 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:57.744979 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:57.745092 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:00:57.746620 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.747897 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.749135 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.749897 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.750025 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:57.752953 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:00:57.752974 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:00:57.752996 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:00:57.753040 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:00:57.757204 (+ 4164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:57.757924 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:57.757933 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:57.757937 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:57.758112 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:57.758837 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:57.758849 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:57.758852 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:57.759328 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:00:57.759676 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:57.759911 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 01:00:57.760163 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 01:00:57.761286 3405 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:00:57.882390 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 3245
I20260810 01:00:57.904583 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 3418
I20260810 01:00:57.918440 3577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:00:57.905503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47552 (local address 127.20.81.131:42639)
0810 01:00:57.905701 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 01:00:57.905706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:00:57.905725 (+ 19us) server_negotiation.cc:407] Connection header received
0810 01:00:57.905884 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:00:57.905888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:00:57.905962 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:00:57.906050 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:00:57.907471 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.908202 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.909465 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:00:57.909845 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:00:57.909957 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:00:57.912586 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:00:57.912598 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:00:57.912612 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:00:57.912659 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:00:57.915543 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:57.916316 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:57.916327 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:57.916329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:57.916494 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:00:57.916910 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:00:57.916913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:00:57.916915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:00:57.917262 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:00:57.917534 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0810 01:00:57.917793 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 01:00:57.917954 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 01:00:57.919131 3554 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:00:58.009110 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 3418
I20260810 01:00:58.029103 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 2983
I20260810 01:00:58.030584 3052 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:00:58.145509 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 2983
I20260810 01:00:58.174100 20806 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 01:00:58.519351 20806 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 01:00:58.519594 20806 mini_ranger.cc:66] Stopping Ranger...
I20260810 01:00:58.865022 20806 mini_ranger.cc:68] Stopped Ranger
2026-08-10 01:00:58.865 UTC [1897] LOG: received smart shutdown request
2026-08-10 01:00:58.869 UTC [1897] LOG: background worker "logical replication launcher" (PID 1906) exited with exit code 1
2026-08-10 01:00:58.869 UTC [1901] LOG: shutting down
2026-08-10 01:00:59.082 UTC [1897] LOG: database system is shut down
2026-08-10T01:00:59Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (54447 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:00:59 dist-test-slave-gp6n krb5kdc[3635](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:00:59 dist-test-slave-gp6n krb5kdc[3635](info): set up 2 sockets
Aug 10 01:00:59 dist-test-slave-gp6n krb5kdc[3635](info): commencing operation
krb5kdc: starting...
W20260810 01:01:00.421341 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:01:00 dist-test-slave-gp6n krb5kdc[3635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:01:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:01:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 01:01:00.718485 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:39581
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:39117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:39581,127.20.81.189:41081,127.20.81.188:36433
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:01:01.067754 3651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:01.068061 3651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:01.068246 3651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:01.075587 3651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:01:01.075691 3651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:01.075798 3651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:01.075898 3651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:01:01.075984 3651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:01:01.086001 3651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:39117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:39581,127.20.81.189:41081,127.20.81.188:36433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:39581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:01.087450 3651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:01.088814 3651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:01:01.101517 3656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:01.101943 3659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:01.101647 3657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:01.102841 3651 server_base.cc:1034] running on GCE node
I20260810 01:01:01.103812 3651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:01.105361 3651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:01.106594 3651 hybrid_clock.cc:648] HybridClock initialized: now 1786323661106557 us; error 30 us; skew 500 ppm
I20260810 01:01:01.106973 3651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:01 dist-test-slave-gp6n krb5kdc[3635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323661, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:01.114492 3651 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 01:01:01.116096 3651 webserver.cc:460] Webserver started at http://127.20.81.190:38371/ using document root <none> and password file <none>
I20260810 01:01:01.116833 3651 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:01.116988 3651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:01.117306 3651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:01:01.120404 3651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb"
format_stamp: "Formatted at 2026-08-10 01:01:01 on dist-test-slave-gp6n"
server_key: "8ee2e1782cd6e1cc4dbb992b137dcc67"
server_key_iv: "38c0f0830a0e13b4b78483bc8594b0a8"
server_key_version: "encryptionkey@0"
I20260810 01:01:01.121259 3651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb"
format_stamp: "Formatted at 2026-08-10 01:01:01 on dist-test-slave-gp6n"
server_key: "8ee2e1782cd6e1cc4dbb992b137dcc67"
server_key_iv: "38c0f0830a0e13b4b78483bc8594b0a8"
server_key_version: "encryptionkey@0"
I20260810 01:01:01.126942 3651 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 01:01:01.130640 3667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:01.132052 3651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 01:01:01.132248 3651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb"
format_stamp: "Formatted at 2026-08-10 01:01:01 on dist-test-slave-gp6n"
server_key: "8ee2e1782cd6e1cc4dbb992b137dcc67"
server_key_iv: "38c0f0830a0e13b4b78483bc8594b0a8"
server_key_version: "encryptionkey@0"
I20260810 01:01:01.132462 3651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:01.157707 3651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:01.162164 3651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:01.162566 3651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:01.189459 3651 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:39581
I20260810 01:01:01.189494 3722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:39581 every 8 connection(s)
I20260810 01:01:01.190977 3651 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:01:01.194216 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 3651
I20260810 01:01:01.194551 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:01:01.194976 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:01:01.195312 20806 external_mini_cluster.cc:1428] Setting key a4c8cb5206fccbe66791b3013957e64d
I20260810 01:01:01.201586 3724 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 01:01:01 dist-test-slave-gp6n krb5kdc[3635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323661, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.81.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.189@KRBTEST.COM" created.
I20260810 01:01:01.227519 3726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.204855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53052 (local address 127.20.81.190:39581)
0810 01:01:01.205729 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:01.205747 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:01.206328 (+ 581us) server_negotiation.cc:407] Connection header received
0810 01:01:01.209307 (+ 2979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:01.209315 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:01.209564 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:01.209747 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:01.211569 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:01.212799 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:01.215126 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:01.215699 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:01.215854 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:01.220114 (+ 4260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:01.220122 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:01.220134 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:01.220163 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:01.223077 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:01.223817 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:01.223827 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:01.223832 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:01.224038 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:01.224476 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:01.224480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:01.224482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:01.225038 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:01.225195 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:01.225835 (+ 640us) server_negotiation.cc:299] Negotiation successful
0810 01:01:01.226366 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":221,"server-negotiator.queue_time_us":627,"thread_start_us":411,"threads_started":1}
I20260810 01:01:01.227622 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.204862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:39581 (local address 127.0.0.1:53052)
0810 01:01:01.206127 (+ 1265us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.206180 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:01.208872 (+ 2692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:01.209862 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:01.209871 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:01.210751 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:01:01.211329 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:01.211356 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:01.212982 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:01.212990 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:01.214953 (+ 1963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:01.214971 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:01.215976 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:01.215986 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:01.216202 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:01.216931 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:01:01.216966 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:01:01.219865 (+ 2899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:01:01.223210 (+ 3345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:01.223224 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:01.223235 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:01.223630 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:01.224153 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:01.224156 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:01.224158 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:01.224343 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:01.225173 (+ 830us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:01:01.225185 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:01:01.225401 (+ 216us) client_negotiation.cc:769] Sending connection context
0810 01:01:01.225631 (+ 230us) client_negotiation.cc:240] Negotiation successful
0810 01:01:01.226175 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1017,"spinlock_wait_cycles":15744,"thread_start_us":804,"threads_started":1}
I20260810 01:01:01.230595 3724 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } has no permanent_uuid. Determining permanent_uuid...
I20260810 01:01:01.231889 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.231364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:41081 (local address 127.0.0.1:35130)
0810 01:01:01.231576 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.231715 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260810 01:01:01.232200 3669 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.81.189:41081: connect: Connection refused (error 111)
I20260810 01:01:01.234061 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.233592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:41081 (local address 127.0.0.1:35138)
0810 01:01:01.233775 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.233877 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260810 01:01:01.234354 3724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.189:41081: Network error: Client connection negotiation failed: client connection to 127.20.81.189:41081: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.189.keytab.
Entry for principal kudu/127.20.81.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.189.keytab.
I20260810 01:01:01.263348 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.189:41081
--webserver_interface=127.20.81.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.189.keytab
--principal=kudu/127.20.81.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:39117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:39581,127.20.81.189:41081,127.20.81.188:36433
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 01:01:01.283833 3724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } attempt: 1
I20260810 01:01:01.285259 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.284694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:41081 (local address 127.0.0.1:35142)
0810 01:01:01.284937 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.285114 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 01:01:01.286557 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.286066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:41081 (local address 127.0.0.1:35156)
0810 01:01:01.286235 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.286316 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260810 01:01:01.286774 3724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.189:41081: Network error: Client connection negotiation failed: client connection to 127.20.81.189:41081: connect: Connection refused (error 111)
I20260810 01:01:01.355026 3724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } attempt: 2
I20260810 01:01:01.356353 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.355758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:41081 (local address 127.0.0.1:60812)
0810 01:01:01.356055 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.356154 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260810 01:01:01.357403 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.357037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:41081 (local address 127.0.0.1:60816)
0810 01:01:01.357211 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.357274 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 01:01:01.357606 3724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.189:41081: Network error: Client connection negotiation failed: client connection to 127.20.81.189:41081: connect: Connection refused (error 111)
I20260810 01:01:01.448894 3724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } attempt: 3
I20260810 01:01:01.450124 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.449660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:41081 (local address 127.0.0.1:60832)
0810 01:01:01.449875 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.449982 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260810 01:01:01.451242 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.450861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:41081 (local address 127.0.0.1:60840)
0810 01:01:01.451018 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.451114 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 01:01:01.451450 3724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.189:41081: Network error: Client connection negotiation failed: client connection to 127.20.81.189:41081: connect: Connection refused (error 111)
I20260810 01:01:01.594755 3724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } attempt: 4
I20260810 01:01:01.596249 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.595657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:41081 (local address 127.0.0.1:60846)
0810 01:01:01.596022 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.596113 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220}
I20260810 01:01:01.597326 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.596939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:41081 (local address 127.0.0.1:60856)
0810 01:01:01.597095 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.597198 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 01:01:01.597512 3724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.189:41081: Network error: Client connection negotiation failed: client connection to 127.20.81.189:41081: connect: Connection refused (error 111)
W20260810 01:01:01.603132 3732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:01.603470 3732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:01.603633 3732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:01.611167 3732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:01:01.611316 3732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:01.611425 3732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:01.611537 3732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:01:01.611620 3732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:01:01.621682 3732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:39117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:39581,127.20.81.189:41081,127.20.81.188:36433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.189.keytab
--principal=kudu/127.20.81.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.189:41081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.81.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:01.623129 3732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:01.624475 3732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:01:01.636790 3738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:01.637303 3739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:01.637548 3732 server_base.cc:1034] running on GCE node
W20260810 01:01:01.637637 3742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:01.638500 3732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:01.639976 3732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:01.641212 3732 hybrid_clock.cc:648] HybridClock initialized: now 1786323661641185 us; error 51 us; skew 500 ppm
I20260810 01:01:01.641602 3732 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:01 dist-test-slave-gp6n krb5kdc[3635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323661, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:01.644963 3732 init.cc:377] Logged in from keytab as kudu/127.20.81.189@KRBTEST.COM (short username kudu)
I20260810 01:01:01.646538 3732 webserver.cc:460] Webserver started at http://127.20.81.189:33933/ using document root <none> and password file <none>
I20260810 01:01:01.647279 3732 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:01.647439 3732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:01.647759 3732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:01:01.650990 3732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-1/data/instance:
uuid: "8848fa9c140b42588b7746e21a2aa31c"
format_stamp: "Formatted at 2026-08-10 01:01:01 on dist-test-slave-gp6n"
server_key: "38b9536c3a25d48d732808c2c30c7948"
server_key_iv: "5cc6fec35d44980382a9041ed69cf0f3"
server_key_version: "encryptionkey@0"
I20260810 01:01:01.651878 3732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-1/wal/instance:
uuid: "8848fa9c140b42588b7746e21a2aa31c"
format_stamp: "Formatted at 2026-08-10 01:01:01 on dist-test-slave-gp6n"
server_key: "38b9536c3a25d48d732808c2c30c7948"
server_key_iv: "5cc6fec35d44980382a9041ed69cf0f3"
server_key_version: "encryptionkey@0"
I20260810 01:01:01.657410 3732 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 01:01:01.661002 3749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:01.662508 3732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 01:01:01.662703 3732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-1/wal
uuid: "8848fa9c140b42588b7746e21a2aa31c"
format_stamp: "Formatted at 2026-08-10 01:01:01 on dist-test-slave-gp6n"
server_key: "38b9536c3a25d48d732808c2c30c7948"
server_key_iv: "5cc6fec35d44980382a9041ed69cf0f3"
server_key_version: "encryptionkey@0"
I20260810 01:01:01.662928 3732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:01.683539 3732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:01.688166 3732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:01.688472 3732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:01.708549 3732 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.189:41081
I20260810 01:01:01.708586 3807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.189:41081 every 8 connection(s)
I20260810 01:01:01.710028 3732 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-1/data/info.pb
I20260810 01:01:01.712065 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 3732
I20260810 01:01:01.712287 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-1/wal/instance
I20260810 01:01:01.712852 20806 external_mini_cluster.cc:1428] Setting key 12937946100ffea7590222e8e9265362
I20260810 01:01:01.714895 3808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:01:01.725032 3808 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 01:01:01 dist-test-slave-gp6n krb5kdc[3635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323661, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.189@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.81.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.188@KRBTEST.COM" created.
I20260810 01:01:01.748206 3726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.727879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52110 (local address 127.20.81.190:39581)
0810 01:01:01.728146 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:01.728155 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:01.729511 (+ 1356us) server_negotiation.cc:407] Connection header received
0810 01:01:01.731541 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:01.731550 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:01.731651 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:01.731779 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:01.733442 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:01.734562 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:01.736334 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:01.737186 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:01.737374 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:01.741093 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:01.741101 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:01.741108 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:01.741159 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:01.744291 (+ 3132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:01.745420 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:01.745426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:01.745430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:01.745605 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:01.746096 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:01.746103 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:01.746109 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:01.746603 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:01.746766 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:01.747512 (+ 746us) server_negotiation.cc:299] Negotiation successful
0810 01:01:01.747779 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 01:01:01.748939 3811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.728394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:39581 (local address 127.0.0.1:52110)
0810 01:01:01.729260 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.729354 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:01.731233 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:01.731999 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:01.732008 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:01.732611 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:01:01.733210 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:01.733228 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:01.734762 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:01.734766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:01.736181 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:01.736194 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:01.737473 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:01.737480 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:01.737730 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:01.738518 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:01:01.738542 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:01:01.740829 (+ 2287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:01:01.744449 (+ 3620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:01.744456 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:01.744470 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:01.745239 (+ 769us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:01.745764 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:01.745768 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:01.745771 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:01.745966 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:01.746740 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:01:01.746746 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:01:01.747112 (+ 366us) client_negotiation.cc:769] Sending connection context
0810 01:01:01.747355 (+ 243us) client_negotiation.cc:240] Negotiation successful
0810 01:01:01.747757 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":399,"threads_started":1}
I20260810 01:01:01.750854 3808 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 01:01:01 dist-test-slave-gp6n krb5kdc[3635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323661, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.189@KRBTEST.COM for kudu/127.20.81.189@KRBTEST.COM
I20260810 01:01:01.768007 3811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.751697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:41081 (local address 127.0.0.1:60860)
0810 01:01:01.752002 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.752028 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:01.752141 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:01.753323 (+ 1182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:01.753331 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:01.753859 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:01:01.754088 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:01.754104 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:01.755571 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:01.755579 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:01.756741 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:01.756753 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:01.757984 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:01.757988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:01.758140 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:01.758822 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:01:01.758840 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:01:01.760957 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:01:01.765458 (+ 4501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:01.765462 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:01.765465 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:01.765819 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:01.766280 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:01.766284 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:01.766286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:01.766512 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:01.767277 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:01:01.767280 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:01:01.767396 (+ 116us) client_negotiation.cc:769] Sending connection context
0810 01:01:01.767529 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 01:01:01.767735 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":3}
I20260810 01:01:01.768571 3812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.752114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60860 (local address 127.20.81.189:41081)
0810 01:01:01.752882 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:01.752889 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:01.752916 (+ 27us) server_negotiation.cc:407] Connection header received
0810 01:01:01.752960 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:01.752965 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:01.753144 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:01.753259 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:01.754255 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:01.755423 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:01.756893 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:01.757842 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:01.757984 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:01.761176 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:01.761193 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:01.761214 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:01.761262 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:01.765296 (+ 4034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:01.765968 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:01.765980 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:01.765986 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:01.766161 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:01.766624 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:01.766632 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:01.766638 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:01.767142 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:01.767365 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:01.767728 (+ 363us) server_negotiation.cc:299] Negotiation successful
0810 01:01:01.768021 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":607,"thread_start_us":449,"threads_started":1}
I20260810 01:01:01.771091 3808 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } has no permanent_uuid. Determining permanent_uuid...
I20260810 01:01:01.772274 3811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.771814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:36433 (local address 127.0.0.1:55264)
0810 01:01:01.772017 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.772120 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260810 01:01:01.772532 3753 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
I20260810 01:01:01.774149 3811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.773822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:36433 (local address 127.0.0.1:55268)
0810 01:01:01.773967 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.774050 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 01:01:01.774364 3808 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:36433: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.188.keytab.
Entry for principal kudu/127.20.81.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.188.keytab.
I20260810 01:01:01.787341 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.188:36433
--webserver_interface=127.20.81.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.188.keytab
--principal=kudu/127.20.81.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:39117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:39581,127.20.81.189:41081,127.20.81.188:36433
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 01:01:01.823757 3808 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } attempt: 1
I20260810 01:01:01.825093 3811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.824532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:36433 (local address 127.0.0.1:55274)
0810 01:01:01.824767 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.824900 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260810 01:01:01.826192 3811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.825797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:36433 (local address 127.0.0.1:55286)
0810 01:01:01.825976 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.826059 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 01:01:01.826514 3808 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:36433: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
I20260810 01:01:01.894793 3808 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } attempt: 2
I20260810 01:01:01.896132 3811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.895638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:36433 (local address 127.0.0.1:55292)
0810 01:01:01.895857 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.895958 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260810 01:01:01.896785 3724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } attempt: 5
I20260810 01:01:01.897464 3811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.896993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:36433 (local address 127.0.0.1:55300)
0810 01:01:01.897165 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.897265 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260810 01:01:01.897754 3808 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:36433: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
Aug 10 01:01:01 dist-test-slave-gp6n krb5kdc[3635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323661, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for kudu/127.20.81.189@KRBTEST.COM
I20260810 01:01:01.912453 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.897360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:41081 (local address 127.0.0.1:60862)
0810 01:01:01.897651 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.897669 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:01.897798 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:01.898203 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:01.898208 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:01.898760 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:01:01.899010 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:01.899024 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:01.900205 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:01.900210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:01.901595 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:01.901610 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:01.902351 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:01.902357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:01.902583 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:01.903235 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:01:01.903258 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:01:01.905750 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:01:01.910075 (+ 4325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:01.910079 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:01.910082 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:01.910537 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:01.910937 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:01.910941 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:01.910943 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:01.911071 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:01.911701 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:01:01.911704 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:01:01.911833 (+ 129us) client_negotiation.cc:769] Sending connection context
0810 01:01:01.911961 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 01:01:01.912172 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172}
I20260810 01:01:01.912870 3812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.897437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60862 (local address 127.20.81.189:41081)
0810 01:01:01.897598 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:01.897606 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:01.897755 (+ 149us) server_negotiation.cc:407] Connection header received
0810 01:01:01.897974 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:01.897978 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:01.898039 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:01.898126 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:01.899211 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:01.900058 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:01.901751 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:01.902177 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:01.902275 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:01.905984 (+ 3709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:01.905993 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:01.905998 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:01.906048 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:01.909932 (+ 3884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:01.910657 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:01.910662 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:01.910666 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:01.910822 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:01.911193 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:01.911198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:01.911201 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:01.911591 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:01.911740 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:01.912130 (+ 390us) server_negotiation.cc:299] Negotiation successful
0810 01:01:01.912362 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 01:01:01.913863 3724 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } has no permanent_uuid. Determining permanent_uuid...
I20260810 01:01:01.914913 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.914544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:36433 (local address 127.0.0.1:55306)
0810 01:01:01.914701 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.914775 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 01:01:01.915887 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.915556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:36433 (local address 127.0.0.1:55322)
0810 01:01:01.915694 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.915784 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 01:01:01.916117 3724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:36433: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
I20260810 01:01:01.967376 3724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } attempt: 1
I20260810 01:01:01.968827 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.968215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:36433 (local address 127.0.0.1:55334)
0810 01:01:01.968479 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.968609 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260810 01:01:01.970033 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.969606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:36433 (local address 127.0.0.1:55338)
0810 01:01:01.969784 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.969892 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260810 01:01:01.970322 3724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:36433: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
I20260810 01:01:01.989034 3808 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } attempt: 3
I20260810 01:01:01.990363 3811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.989871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:36433 (local address 127.0.0.1:55346)
0810 01:01:01.990089 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.990191 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 01:01:01.991714 3811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:01.991177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:36433 (local address 127.0.0.1:55354)
0810 01:01:01.991497 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:01.991594 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195}
W20260810 01:01:01.991990 3808 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:36433: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
I20260810 01:01:02.038594 3724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } attempt: 2
I20260810 01:01:02.039975 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:02.039316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:36433 (local address 127.0.0.1:55370)
0810 01:01:02.039615 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:02.039741 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260810 01:01:02.041213 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:02.040769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:36433 (local address 127.0.0.1:55374)
0810 01:01:02.040958 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:02.041059 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260810 01:01:02.041525 3724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:36433: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
I20260810 01:01:02.135298 3808 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } attempt: 4
I20260810 01:01:02.136708 3811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:02.136056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:36433 (local address 127.0.0.1:55378)
0810 01:01:02.136328 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:02.136453 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260810 01:01:02.137872 3811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:02.137520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:36433 (local address 127.0.0.1:55388)
0810 01:01:02.137668 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:02.137746 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 01:01:02.138132 3808 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:36433: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
I20260810 01:01:02.147773 3724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } attempt: 3
I20260810 01:01:02.148937 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:02.148393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:36433 (local address 127.0.0.1:55400)
0810 01:01:02.148627 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:02.148732 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 01:01:02.150028 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:02.149639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:36433 (local address 127.0.0.1:55414)
0810 01:01:02.149805 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:02.149891 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260810 01:01:02.150260 3724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:36433: Network error: Client connection negotiation failed: client connection to 127.20.81.188:36433: connect: Connection refused (error 111)
W20260810 01:01:02.151695 3815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:02.152026 3815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:02.152192 3815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:02.159768 3815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:01:02.159893 3815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:02.159976 3815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:02.160085 3815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:01:02.160173 3815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:01:02.170964 3815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:39117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:39581,127.20.81.189:41081,127.20.81.188:36433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.188.keytab
--principal=kudu/127.20.81.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.188:36433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.81.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:02.172370 3815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:02.173769 3815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:01:02.186218 3821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:02.186553 3820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:02.187091 3815 server_base.cc:1034] running on GCE node
W20260810 01:01:02.186105 3823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:02.188210 3815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:02.189713 3815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:02.191051 3815 hybrid_clock.cc:648] HybridClock initialized: now 1786323662190983 us; error 60 us; skew 500 ppm
I20260810 01:01:02.191452 3815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:02 dist-test-slave-gp6n krb5kdc[3635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323662, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:02.195605 3815 init.cc:377] Logged in from keytab as kudu/127.20.81.188@KRBTEST.COM (short username kudu)
I20260810 01:01:02.197214 3815 webserver.cc:460] Webserver started at http://127.20.81.188:39979/ using document root <none> and password file <none>
I20260810 01:01:02.197918 3815 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:02.198071 3815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:02.198381 3815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:01:02.201442 3815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-2/data/instance:
uuid: "b10f37a1444b42d4a783af3150d05980"
format_stamp: "Formatted at 2026-08-10 01:01:02 on dist-test-slave-gp6n"
server_key: "b3a47da3e85a35837eb07110d2610736"
server_key_iv: "b54efaa7c7ab7536e52b1147e80cdd64"
server_key_version: "encryptionkey@0"
I20260810 01:01:02.202286 3815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-2/wal/instance:
uuid: "b10f37a1444b42d4a783af3150d05980"
format_stamp: "Formatted at 2026-08-10 01:01:02 on dist-test-slave-gp6n"
server_key: "b3a47da3e85a35837eb07110d2610736"
server_key_iv: "b54efaa7c7ab7536e52b1147e80cdd64"
server_key_version: "encryptionkey@0"
I20260810 01:01:02.207717 3815 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 01:01:02.211457 3830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:02.212854 3815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 01:01:02.213057 3815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-2/wal
uuid: "b10f37a1444b42d4a783af3150d05980"
format_stamp: "Formatted at 2026-08-10 01:01:02 on dist-test-slave-gp6n"
server_key: "b3a47da3e85a35837eb07110d2610736"
server_key_iv: "b54efaa7c7ab7536e52b1147e80cdd64"
server_key_version: "encryptionkey@0"
I20260810 01:01:02.213272 3815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:02.233924 3815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:02.238933 3815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:02.239253 3815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:02.259402 3815 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.188:36433
I20260810 01:01:02.259456 3886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.188:36433 every 8 connection(s)
I20260810 01:01:02.260944 3815 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-2/data/info.pb
I20260810 01:01:02.264950 3887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:01:02.266904 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 3815
I20260810 01:01:02.267163 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323445691443-20806-0/minicluster-data/master-2/wal/instance
I20260810 01:01:02.267776 20806 external_mini_cluster.cc:1428] Setting key 998e5789c2701fa9549a5b3af84b2d1c
I20260810 01:01:02.274127 3887 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 01:01:02 dist-test-slave-gp6n krb5kdc[3635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.188@KRBTEST.COM
Aug 10 01:01:02 dist-test-slave-gp6n krb5kdc[3635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323662, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.188@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:01:02.294243 3892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:02.277210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52114 (local address 127.20.81.190:39581)
0810 01:01:02.277790 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:02.277796 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:02.278494 (+ 698us) server_negotiation.cc:407] Connection header received
0810 01:01:02.278561 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:02.278564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:02.278625 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:02.278750 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:02.280046 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.281076 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.282619 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.283375 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.283528 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:02.287716 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:02.287722 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:02.287726 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:02.287763 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:02.290972 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:02.291823 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:02.291828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:02.291833 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:02.292001 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:02.292467 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:02.292473 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:02.292477 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:02.292828 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:02.292956 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:02.293565 (+ 609us) server_negotiation.cc:299] Negotiation successful
0810 01:01:02.293749 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":270,"threads_started":1}
I20260810 01:01:02.294763 3891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:02.269864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55424 (local address 127.20.81.188:36433)
0810 01:01:02.270890 (+ 1026us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:02.270908 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:02.270971 (+ 63us) server_negotiation.cc:407] Connection header received
0810 01:01:02.273383 (+ 2412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:02.273412 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:02.273919 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:02.274375 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:02.275895 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.278063 (+ 2168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.279574 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.280414 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.280551 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:02.285553 (+ 5002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:02.285565 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:02.285588 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:02.285641 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:02.288957 (+ 3316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:02.290063 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:02.290070 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:02.290078 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:02.290265 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:02.290753 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:02.290756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:02.290758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:02.291576 (+ 818us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:02.291893 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:02.292476 (+ 583us) server_negotiation.cc:299] Negotiation successful
0810 01:01:02.292970 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":443,"threads_started":1}
I20260810 01:01:02.294790 3894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:02.277190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:39581 (local address 127.0.0.1:52114)
0810 01:01:02.278164 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:02.278212 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:02.278351 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:02.278821 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:02.278828 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:02.279451 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:01:02.279877 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:02.279888 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.281267 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.281273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:02.282395 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:02.282476 (+ 81us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.283606 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.283611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:02.283732 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:02.284561 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:01:02.284594 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:01:02.287464 (+ 2870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:01:02.291107 (+ 3643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:02.291115 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:02.291121 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:02.291701 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:02.292147 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:02.292155 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:02.292160 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:02.292318 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:02.292967 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:01:02.292975 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:01:02.293174 (+ 199us) client_negotiation.cc:769] Sending connection context
0810 01:01:02.293393 (+ 219us) client_negotiation.cc:240] Negotiation successful
0810 01:01:02.293710 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"thread_start_us":543,"threads_started":1}
I20260810 01:01:02.296491 3887 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 01:01:02 dist-test-slave-gp6n krb5kdc[3635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323662, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.188@KRBTEST.COM for kudu/127.20.81.189@KRBTEST.COM
Aug 10 01:01:02 dist-test-slave-gp6n krb5kdc[3635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:01:02.311866 3812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:02.297444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60872 (local address 127.20.81.189:41081)
0810 01:01:02.297655 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:02.297661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:02.297816 (+ 155us) server_negotiation.cc:407] Connection header received
0810 01:01:02.298018 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:02.298047 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:02.298137 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:02.298263 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:02.299205 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.299899 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.301276 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.301711 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.301804 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:02.305542 (+ 3738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:02.305547 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:02.305551 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:02.305584 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:02.309001 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:02.309857 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:02.309865 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:02.309878 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:02.310058 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:02.310469 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:02.310474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:02.310478 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:02.310837 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:02.311015 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:02.311378 (+ 363us) server_negotiation.cc:299] Negotiation successful
0810 01:01:02.311602 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 01:01:02.311890 3894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:02.297468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:41081 (local address 127.0.0.1:60872)
0810 01:01:02.297724 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:02.297747 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:02.297874 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:02.298283 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:02.298288 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:02.298864 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:01:02.299058 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:02.299068 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.300074 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.300079 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:02.301133 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:02.301148 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.301863 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.301870 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:02.302084 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:02.302801 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:01:02.302818 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:01:02.305267 (+ 2449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:01:02.309154 (+ 3887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:02.309158 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:02.309161 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:02.309725 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:02.310183 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:02.310187 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:02.310189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:02.310328 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:02.310972 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:01:02.310975 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:01:02.311121 (+ 146us) client_negotiation.cc:769] Sending connection context
0810 01:01:02.311235 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 01:01:02.311528 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
I20260810 01:01:02.313118 3887 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } has no permanent_uuid. Determining permanent_uuid...
I20260810 01:01:02.315021 3892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:02.298364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52124 (local address 127.20.81.190:39581)
0810 01:01:02.298584 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:02.298589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:02.298607 (+ 18us) server_negotiation.cc:407] Connection header received
0810 01:01:02.298679 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:02.298687 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:02.298756 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:02.298863 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:02.300039 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.300713 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.302315 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.302947 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.303043 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:02.307809 (+ 4766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:02.307818 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:02.307823 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:02.307867 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:02.311343 (+ 3476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:02.312488 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:02.312496 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:02.312499 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:02.312658 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:02.313160 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:02.313164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:02.313166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:02.313508 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:02.313815 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:02.314380 (+ 565us) server_negotiation.cc:299] Negotiation successful
0810 01:01:02.314680 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
Aug 10 01:01:02 dist-test-slave-gp6n krb5kdc[3635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323662, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.188@KRBTEST.COM for kudu/127.20.81.188@KRBTEST.COM
Aug 10 01:01:02 dist-test-slave-gp6n krb5kdc[3635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.189@KRBTEST.COM
I20260810 01:01:02.327162 3891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:02.313932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55426 (local address 127.20.81.188:36433)
0810 01:01:02.314095 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:02.314100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:02.314114 (+ 14us) server_negotiation.cc:407] Connection header received
0810 01:01:02.314264 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:02.314271 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:02.314344 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:02.314502 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:02.315304 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.315980 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.317528 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.318074 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.318219 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:02.321483 (+ 3264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:02.321488 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:02.321491 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:02.321518 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:02.324342 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:02.325100 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:02.325104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:02.325106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:02.325228 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:02.325628 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:02.325635 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:02.325639 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:02.325919 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:02.326080 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:02.326465 (+ 385us) server_negotiation.cc:299] Negotiation successful
0810 01:01:02.326753 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":64}
I20260810 01:01:02.327268 3894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:02.313830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:36433 (local address 127.0.0.1:55426)
0810 01:01:02.314025 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:02.314045 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:02.314144 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:02.314535 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:02.314542 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:02.315001 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:01:02.315174 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:02.315186 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.316218 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.316222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:02.317383 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:02.317401 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.318205 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.318210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:02.318451 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:02.319081 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:01:02.319098 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:01:02.321285 (+ 2187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:01:02.324504 (+ 3219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:02.324509 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:02.324513 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:02.324972 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:02.325349 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:02.325354 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:02.325358 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:02.325533 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:02.326039 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:01:02.326044 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:01:02.326195 (+ 151us) client_negotiation.cc:769] Sending connection context
0810 01:01:02.326355 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 01:01:02.326905 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":164}
I20260810 01:01:02.327502 3724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } attempt: 4
I20260810 01:01:02.333140 3812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:02.316937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60886 (local address 127.20.81.189:41081)
0810 01:01:02.317083 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:02.317110 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:02.317130 (+ 20us) server_negotiation.cc:407] Connection header received
0810 01:01:02.317332 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:02.317336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:02.317416 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:02.317557 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:02.319312 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.320012 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.321461 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.321877 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.322030 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:02.326463 (+ 4433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:02.326470 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:02.326475 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:02.326519 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:02.330050 (+ 3531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:02.331016 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:02.331021 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:02.331025 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:02.331209 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:02.331652 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:02.331657 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:02.331660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:02.331940 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:02.332251 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:02.332542 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 01:01:02.332788 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 01:01:02.334236 3887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980: Bootstrap starting.
Aug 10 01:01:02 dist-test-slave-gp6n krb5kdc[3635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323661, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for kudu/127.20.81.188@KRBTEST.COM
I20260810 01:01:02.338387 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 01:01:02.338867 3887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980: Neither blocks nor log segments found. Creating new log.
I20260810 01:01:02.340090 3887 log.cc:824] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980: Log is configured to *not* fsync() on all Append() calls
I20260810 01:01:02.343206 3887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980: No bootstrap required, opened a new log
I20260810 01:01:02.345649 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:02.328215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:36433 (local address 127.0.0.1:55442)
0810 01:01:02.328511 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:02.328533 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:02.328647 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:02.329026 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:02.329032 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:02.329596 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:01:02.329811 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:02.329823 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.331155 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.331160 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:02.332670 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:02.332686 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.333870 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.333876 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:02.334074 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:02.334759 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:01:02.334783 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:01:02.337512 (+ 2729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:01:02.341822 (+ 4310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:02.341831 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:02.341835 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:02.342451 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:02.343049 (+ 598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:02.343057 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:02.343064 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:02.343292 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:02.344240 (+ 948us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:01:02.344248 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:01:02.344506 (+ 258us) client_negotiation.cc:769] Sending connection context
0810 01:01:02.344668 (+ 162us) client_negotiation.cc:240] Negotiation successful
0810 01:01:02.345064 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169}
I20260810 01:01:02.346948 3891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:02.328319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55442 (local address 127.20.81.188:36433)
0810 01:01:02.328697 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:02.328705 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:02.328731 (+ 26us) server_negotiation.cc:407] Connection header received
0810 01:01:02.328783 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:02.328791 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:02.328860 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:02.328962 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:02.329988 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.330888 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.332846 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.333680 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.334466 (+ 786us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:02.337731 (+ 3265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:02.337738 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:02.337744 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:02.337788 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:02.341648 (+ 3860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:02.342657 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:02.342666 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:02.342672 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:02.342885 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:02.343482 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:02.343490 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:02.343496 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:02.344035 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:02.346011 (+ 1976us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:02.346198 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 01:01:02.346486 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 01:01:02.352880 3724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb: Bootstrap starting.
I20260810 01:01:02.353554 3887 raft_consensus.cc:348] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 } } peers { permanent_uuid: "8848fa9c140b42588b7746e21a2aa31c" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } } peers { permanent_uuid: "b10f37a1444b42d4a783af3150d05980" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } }
I20260810 01:01:02.354019 3887 raft_consensus.cc:374] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:01:02.354146 3887 raft_consensus.cc:729] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10f37a1444b42d4a783af3150d05980, State: Initialized, Role: FOLLOWER
I20260810 01:01:02.354936 3887 consensus_queue.cc:260] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 } } peers { permanent_uuid: "8848fa9c140b42588b7746e21a2aa31c" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } } peers { permanent_uuid: "b10f37a1444b42d4a783af3150d05980" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } }
I20260810 01:01:02.358130 3724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb: Neither blocks nor log segments found. Creating new log.
I20260810 01:01:02.359925 3724 log.cc:824] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb: Log is configured to *not* fsync() on all Append() calls
I20260810 01:01:02.362083 3902 sys_catalog.cc:455] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 } } peers { permanent_uuid: "8848fa9c140b42588b7746e21a2aa31c" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } } peers { permanent_uuid: "b10f37a1444b42d4a783af3150d05980" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } } }
I20260810 01:01:02.362620 3902 sys_catalog.cc:458] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:01:02.364857 3887 sys_catalog.cc:564] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:01:02.372263 3724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb: No bootstrap required, opened a new log
I20260810 01:01:02.385840 3812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:02.349030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60896 (local address 127.20.81.189:41081)
0810 01:01:02.353581 (+ 4551us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:02.353590 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:02.367653 (+ 14063us) server_negotiation.cc:407] Connection header received
0810 01:01:02.367856 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:02.367863 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:02.367959 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:02.368061 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:02.370483 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.371611 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.373686 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.374424 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.374542 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:02.378540 (+ 3998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:02.378566 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:02.378576 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:02.378626 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:02.382291 (+ 3665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:02.383324 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:02.383329 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:02.383332 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:02.383583 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:02.384041 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:02.384046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:02.384049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:02.384443 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:02.384664 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:02.385211 (+ 547us) server_negotiation.cc:299] Negotiation successful
0810 01:01:02.385451 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4366}
I20260810 01:01:02.385840 3892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:02.349238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52128 (local address 127.20.81.190:39581)
0810 01:01:02.349504 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:02.349513 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:02.360049 (+ 10536us) server_negotiation.cc:407] Connection header received
0810 01:01:02.361811 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:02.361820 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:02.361929 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:02.362036 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:02.368459 (+ 6423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.369771 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.372125 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.372820 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.372923 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:02.377673 (+ 4750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:02.377682 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:02.377687 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:02.377736 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:02.381995 (+ 4259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:02.383142 (+ 1147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:02.383149 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:02.383152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:02.383318 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:02.383770 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:02.383778 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:02.383784 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:02.384198 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:02.384444 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:02.385143 (+ 699us) server_negotiation.cc:299] Negotiation successful
0810 01:01:02.385395 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260810 01:01:02.388175 3891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:02.360831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55452 (local address 127.20.81.188:36433)
0810 01:01:02.360986 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:02.360992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:02.368147 (+ 7155us) server_negotiation.cc:407] Connection header received
0810 01:01:02.368439 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:02.368445 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:02.368549 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:02.368671 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:02.371482 (+ 2811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.372472 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.374856 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.375537 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.375638 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:02.379571 (+ 3933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:02.379580 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:02.379588 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:02.379641 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:02.384146 (+ 4505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:02.385236 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:02.385242 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:02.385246 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:02.385406 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:02.385863 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:02.385868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:02.385871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:02.386694 (+ 823us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:02.386951 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:02.387546 (+ 595us) server_negotiation.cc:299] Negotiation successful
0810 01:01:02.387792 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 01:01:02.390786 3724 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 } } peers { permanent_uuid: "8848fa9c140b42588b7746e21a2aa31c" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } } peers { permanent_uuid: "b10f37a1444b42d4a783af3150d05980" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } }
I20260810 01:01:02.391270 3724 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:01:02.391402 3724 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4fa5f7e5c5467fa0b9e830672e2afb, State: Initialized, Role: FOLLOWER
I20260810 01:01:02.392099 3724 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 } } peers { permanent_uuid: "8848fa9c140b42588b7746e21a2aa31c" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } } peers { permanent_uuid: "b10f37a1444b42d4a783af3150d05980" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } }
I20260810 01:01:02.393110 3911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 } } peers { permanent_uuid: "8848fa9c140b42588b7746e21a2aa31c" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } } peers { permanent_uuid: "b10f37a1444b42d4a783af3150d05980" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } } }
I20260810 01:01:02.393594 3911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:01:02.394753 3724 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb [sys.catalog]: configured and running, proceeding with master startup.
W20260810 01:01:02.402148 3916 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 01:01:02.402446 3916 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 01:01:02.397214 3815 master_runner.cc:186] Error getting master registration for 127.20.81.190:39581: OK, instance_id { permanent_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb" instance_seqno: 1786323661183713 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 01:01:02.408255 3815 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 01:01:02.412773 3929 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 01:01:02.413054 3929 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 01:01:02.415573 3673 tablet.cc:2369] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:01:02.411513 3651 master_runner.cc:186] Error getting master registration for 127.20.81.189:41081: OK, instance_id { permanent_uuid: "8848fa9c140b42588b7746e21a2aa31c" instance_seqno: 1786323661702917 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 01:01:02.420774 3651 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:01:02.437397 3808 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } attempt: 5
Aug 10 01:01:02 dist-test-slave-gp6n krb5kdc[3635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323661, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.189@KRBTEST.COM for kudu/127.20.81.188@KRBTEST.COM
I20260810 01:01:02.452696 3811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:02.438027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:36433 (local address 127.0.0.1:55458)
0810 01:01:02.438278 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:02.438299 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:02.438459 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:02.438816 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:02.438821 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:02.439306 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:01:02.439484 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:02.439493 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.440580 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.440588 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:02.441626 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:02.441638 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.442556 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.442564 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:02.442798 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:02.443468 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:01:02.443486 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:01:02.445914 (+ 2428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:01:02.450024 (+ 4110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:02.450032 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:02.450038 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:02.450523 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:02.450941 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:02.450946 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:02.450950 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:02.451091 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:02.451622 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:01:02.451627 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:01:02.451784 (+ 157us) client_negotiation.cc:769] Sending connection context
0810 01:01:02.451943 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 01:01:02.452262 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260810 01:01:02.452718 3891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:02.438096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55458 (local address 127.20.81.188:36433)
0810 01:01:02.438281 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:02.438288 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:02.438385 (+ 97us) server_negotiation.cc:407] Connection header received
0810 01:01:02.438598 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:02.438604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:02.438675 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:02.438791 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:02.439644 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.440394 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.441818 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:02.442349 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:02.442545 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:02.446149 (+ 3604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:02.446158 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:02.446165 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:02.446211 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:02.449877 (+ 3666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:02.450642 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:02.450649 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:02.450653 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:02.450810 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:02.451208 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:02.451214 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:02.451218 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:02.451497 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:02.451695 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:02.452040 (+ 345us) server_negotiation.cc:299] Negotiation successful
0810 01:01:02.452287 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 01:01:02.457837 3808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c: Bootstrap starting.
I20260810 01:01:02.463048 3808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c: Neither blocks nor log segments found. Creating new log.
I20260810 01:01:02.464188 3808 log.cc:824] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c: Log is configured to *not* fsync() on all Append() calls
I20260810 01:01:02.467543 3808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c: No bootstrap required, opened a new log
I20260810 01:01:02.476828 3808 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 } } peers { permanent_uuid: "8848fa9c140b42588b7746e21a2aa31c" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } } peers { permanent_uuid: "b10f37a1444b42d4a783af3150d05980" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } }
I20260810 01:01:02.477228 3808 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:01:02.477404 3808 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8848fa9c140b42588b7746e21a2aa31c, State: Initialized, Role: FOLLOWER
I20260810 01:01:02.478093 3808 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 } } peers { permanent_uuid: "8848fa9c140b42588b7746e21a2aa31c" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } } peers { permanent_uuid: "b10f37a1444b42d4a783af3150d05980" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } }
I20260810 01:01:02.479107 3933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 } } peers { permanent_uuid: "8848fa9c140b42588b7746e21a2aa31c" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } } peers { permanent_uuid: "b10f37a1444b42d4a783af3150d05980" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } } }
I20260810 01:01:02.479568 3933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:01:02.480368 3808 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c [sys.catalog]: configured and running, proceeding with master startup.
W20260810 01:01:02.491444 3837 tablet.cc:2369] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:01:02.502097 3902 raft_consensus.cc:482] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:01:02.502478 3902 raft_consensus.cc:504] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 } } peers { permanent_uuid: "8848fa9c140b42588b7746e21a2aa31c" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } } peers { permanent_uuid: "b10f37a1444b42d4a783af3150d05980" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } }
I20260810 01:01:02.504235 3902 leader_election.cc:288] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a4fa5f7e5c5467fa0b9e830672e2afb (127.20.81.190:39581), 8848fa9c140b42588b7746e21a2aa31c (127.20.81.189:41081)
I20260810 01:01:02.505163 3732 master_runner.cc:263] No leader master found from master 8848fa9c140b42588b7746e21a2aa31c
I20260810 01:01:02.505332 3732 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:01:02.505983 3781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b10f37a1444b42d4a783af3150d05980" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8848fa9c140b42588b7746e21a2aa31c" is_pre_election: true
I20260810 01:01:02.506652 3781 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b10f37a1444b42d4a783af3150d05980 in term 0.
I20260810 01:01:02.507604 3832 leader_election.cc:302] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8848fa9c140b42588b7746e21a2aa31c, b10f37a1444b42d4a783af3150d05980; no voters:
I20260810 01:01:02.508416 3696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b10f37a1444b42d4a783af3150d05980" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb" is_pre_election: true
I20260810 01:01:02.509050 3696 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b10f37a1444b42d4a783af3150d05980 in term 0.
I20260810 01:01:02.510010 3902 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:01:02.510202 3902 raft_consensus.cc:482] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:01:02.510309 3902 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:01:02.513824 3902 raft_consensus.cc:504] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 } } peers { permanent_uuid: "8848fa9c140b42588b7746e21a2aa31c" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } } peers { permanent_uuid: "b10f37a1444b42d4a783af3150d05980" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } }
I20260810 01:01:02.515118 3696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b10f37a1444b42d4a783af3150d05980" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb"
I20260810 01:01:02.515358 3696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:01:02.519371 3696 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b10f37a1444b42d4a783af3150d05980 in term 1.
I20260810 01:01:02.519964 3833 leader_election.cc:302] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a4fa5f7e5c5467fa0b9e830672e2afb, b10f37a1444b42d4a783af3150d05980; no voters:
I20260810 01:01:02.527345 3902 leader_election.cc:288] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [CANDIDATE]: Term 1 election: Requested vote from peers 9a4fa5f7e5c5467fa0b9e830672e2afb (127.20.81.190:39581), 8848fa9c140b42588b7746e21a2aa31c (127.20.81.189:41081)
I20260810 01:01:02.527402 3781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b10f37a1444b42d4a783af3150d05980" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8848fa9c140b42588b7746e21a2aa31c"
I20260810 01:01:02.527648 3902 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:01:02.527644 3781 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:01:02.527804 3902 raft_consensus.cc:686] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [term 1 LEADER]: Becoming Leader. State: Replica: b10f37a1444b42d4a783af3150d05980, State: Running, Role: LEADER
I20260810 01:01:02.528441 3902 consensus_queue.cc:237] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 } } peers { permanent_uuid: "8848fa9c140b42588b7746e21a2aa31c" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } } peers { permanent_uuid: "b10f37a1444b42d4a783af3150d05980" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } }
I20260810 01:01:02.531472 3781 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b10f37a1444b42d4a783af3150d05980 in term 1.
W20260810 01:01:02.532465 3945 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 01:01:02.532588 3945 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 01:01:02.552035 3948 sys_catalog.cc:455] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b10f37a1444b42d4a783af3150d05980. Latest consensus state: current_term: 1 leader_uuid: "b10f37a1444b42d4a783af3150d05980" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 } } peers { permanent_uuid: "8848fa9c140b42588b7746e21a2aa31c" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } } peers { permanent_uuid: "b10f37a1444b42d4a783af3150d05980" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } } }
I20260810 01:01:02.552513 3948 sys_catalog.cc:458] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [sys.catalog]: This master's current role is: LEADER
I20260810 01:01:02.563095 3950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:01:02.565425 3950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:01:02.587968 3696 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb [term 1 FOLLOWER]: Refusing update from remote peer b10f37a1444b42d4a783af3150d05980: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:01:02.589143 3948 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:01:02.591190 3781 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c [term 1 FOLLOWER]: Refusing update from remote peer b10f37a1444b42d4a783af3150d05980: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:01:02.592813 3948 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8848fa9c140b42588b7746e21a2aa31c" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:01:02.618881 3933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b10f37a1444b42d4a783af3150d05980. Latest consensus state: current_term: 1 leader_uuid: "b10f37a1444b42d4a783af3150d05980" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 } } peers { permanent_uuid: "8848fa9c140b42588b7746e21a2aa31c" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } } peers { permanent_uuid: "b10f37a1444b42d4a783af3150d05980" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } } }
I20260810 01:01:02.619238 3933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:01:02.619737 3911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb [sys.catalog]: SysCatalogTable state changed. Reason: New leader b10f37a1444b42d4a783af3150d05980. Latest consensus state: current_term: 1 leader_uuid: "b10f37a1444b42d4a783af3150d05980" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 } } peers { permanent_uuid: "8848fa9c140b42588b7746e21a2aa31c" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } } peers { permanent_uuid: "b10f37a1444b42d4a783af3150d05980" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } } }
I20260810 01:01:02.619997 3911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:01:02.630326 3952 mvcc.cc:205] Tried to move back new op lower bound from 7316781721948332032 to 7316781721734316032. Current Snapshot: MvccSnapshot[applied={T|T < 7316781721948332032}]
I20260810 01:01:02.634090 3911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b10f37a1444b42d4a783af3150d05980" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 } } peers { permanent_uuid: "8848fa9c140b42588b7746e21a2aa31c" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } } peers { permanent_uuid: "b10f37a1444b42d4a783af3150d05980" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } } }
I20260810 01:01:02.634395 3911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:01:02.639964 3933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b10f37a1444b42d4a783af3150d05980" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 } } peers { permanent_uuid: "8848fa9c140b42588b7746e21a2aa31c" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } } peers { permanent_uuid: "b10f37a1444b42d4a783af3150d05980" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } } }
I20260810 01:01:02.640261 3933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:01:02.643635 3953 mvcc.cc:205] Tried to move back new op lower bound from 7316781721948332032 to 7316781721734316032. Current Snapshot: MvccSnapshot[applied={T|T < 7316781721948332032 or (T in {7316781721948332032})}]
I20260810 01:01:02.647274 3950 catalog_manager.cc:1347] Generated new cluster ID: efaee6c280a74b77b96c9f61d26955f3
I20260810 01:01:02.647397 3950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:01:02.652029 3948 sys_catalog.cc:455] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b10f37a1444b42d4a783af3150d05980" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 } } peers { permanent_uuid: "8848fa9c140b42588b7746e21a2aa31c" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } } peers { permanent_uuid: "b10f37a1444b42d4a783af3150d05980" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } } }
I20260810 01:01:02.652313 3948 sys_catalog.cc:458] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [sys.catalog]: This master's current role is: LEADER
I20260810 01:01:02.652572 3948 sys_catalog.cc:455] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b10f37a1444b42d4a783af3150d05980" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4fa5f7e5c5467fa0b9e830672e2afb" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39581 } } peers { permanent_uuid: "8848fa9c140b42588b7746e21a2aa31c" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 41081 } } peers { permanent_uuid: "b10f37a1444b42d4a783af3150d05980" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 36433 } } }
I20260810 01:01:02.652755 3948 sys_catalog.cc:458] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 [sys.catalog]: This master's current role is: LEADER
I20260810 01:01:02.688280 3950 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:01:02.690577 3950 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:01:02.774098 3950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980: Generated new TSK 0
I20260810 01:01:02.775363 3950 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:01:03.415210 3929 catalog_manager.cc:1259] Loaded cluster ID: efaee6c280a74b77b96c9f61d26955f3
I20260810 01:01:03.415380 3929 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb: loading cluster ID for follower catalog manager: success
I20260810 01:01:03.418087 3929 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb: acquiring CA information for follower catalog manager: success
I20260810 01:01:03.420114 3929 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9a4fa5f7e5c5467fa0b9e830672e2afb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 01:01:03.534750 3945 catalog_manager.cc:1259] Loaded cluster ID: efaee6c280a74b77b96c9f61d26955f3
I20260810 01:01:03.534868 3945 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c: loading cluster ID for follower catalog manager: success
I20260810 01:01:03.537689 3945 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c: acquiring CA information for follower catalog manager: success
I20260810 01:01:03.539528 3945 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8848fa9c140b42588b7746e21a2aa31c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 01:01:03.902585 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 3651
I20260810 01:01:03.904024 3718 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:04.016850 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 3651
I20260810 01:01:04.043385 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 3732
I20260810 01:01:04.044657 3803 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:04.141925 3965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:04.141119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:39581 (local address 127.0.0.1:52142)
0810 01:01:04.141630 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:04.141756 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:39581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":300,"threads_started":1}
W20260810 01:01:04.142242 3833 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.81.190:39581: connect: Connection refused (error 111)
I20260810 01:01:04.144656 3965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:04.144299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:39581 (local address 127.0.0.1:52146)
0810 01:01:04.144445 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:04.144540 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:39581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 01:01:04.144879 3833 consensus_peers.cc:487] T 00000000000000000000000000000000 P b10f37a1444b42d4a783af3150d05980 -> Peer 9a4fa5f7e5c5467fa0b9e830672e2afb (127.20.81.190:39581): Couldn't send request to peer 9a4fa5f7e5c5467fa0b9e830672e2afb. Status: Network error: Client connection negotiation failed: client connection to 127.20.81.190:39581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 01:01:04.156402 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 3732
I20260810 01:01:04.182286 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 3815
I20260810 01:01:04.183531 3882 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:04.300518 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 3815
2026-08-10T01:01:04Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5084 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:01:04 dist-test-slave-gp6n krb5kdc[3970](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:01:04 dist-test-slave-gp6n krb5kdc[3970](info): set up 2 sockets
Aug 10 01:01:04 dist-test-slave-gp6n krb5kdc[3970](info): commencing operation
krb5kdc: starting...
W20260810 01:01:05.520191 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:01:05 dist-test-slave-gp6n krb5kdc[3970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:01:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:01:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 01:01:05.826146 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:40789
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:39855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:40789
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:01:06.182721 3984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:06.183032 3984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:06.183262 3984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:06.190578 3984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:01:06.190721 3984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:06.190820 3984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:06.190927 3984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:01:06.191023 3984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:01:06.202819 3984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:39855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:40789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:40789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:06.204258 3984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:06.205685 3984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:01:06.218565 3993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:06.218662 3992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:06.218717 3995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:06.220374 3984 server_base.cc:1034] running on GCE node
I20260810 01:01:06.221211 3984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:06.222780 3984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:06.224040 3984 hybrid_clock.cc:648] HybridClock initialized: now 1786323666223969 us; error 55 us; skew 500 ppm
I20260810 01:01:06.224467 3984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:06 dist-test-slave-gp6n krb5kdc[3970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323666, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:06.232991 3984 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 01:01:06.234642 3984 webserver.cc:460] Webserver started at http://127.20.81.190:46531/ using document root <none> and password file <none>
I20260810 01:01:06.235356 3984 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:06.235463 3984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:06.235731 3984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:01:06.238891 3984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "e85a013f4e3448ddb042201ee9325255"
format_stamp: "Formatted at 2026-08-10 01:01:06 on dist-test-slave-gp6n"
server_key: "ef0c6c3856b776bc529a4694d6dc00f8"
server_key_iv: "d507721f68efe07a491bafb12534e182"
server_key_version: "encryptionkey@0"
I20260810 01:01:06.239709 3984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "e85a013f4e3448ddb042201ee9325255"
format_stamp: "Formatted at 2026-08-10 01:01:06 on dist-test-slave-gp6n"
server_key: "ef0c6c3856b776bc529a4694d6dc00f8"
server_key_iv: "d507721f68efe07a491bafb12534e182"
server_key_version: "encryptionkey@0"
I20260810 01:01:06.245003 3984 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 01:01:06.248605 4003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:06.249940 3984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 01:01:06.250135 3984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "e85a013f4e3448ddb042201ee9325255"
format_stamp: "Formatted at 2026-08-10 01:01:06 on dist-test-slave-gp6n"
server_key: "ef0c6c3856b776bc529a4694d6dc00f8"
server_key_iv: "d507721f68efe07a491bafb12534e182"
server_key_version: "encryptionkey@0"
I20260810 01:01:06.250346 3984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:06.281307 3984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:06.285857 3984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:06.286162 3984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:06.305198 3984 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:40789
I20260810 01:01:06.305222 4063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:40789 every 8 connection(s)
I20260810 01:01:06.306718 3984 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:01:06.307928 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 3984
I20260810 01:01:06.308183 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:01:06.309105 20806 external_mini_cluster.cc:1428] Setting key c52646127c9d5c9678b06cbefcf62ad2
I20260810 01:01:06.311866 4064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:01:06.325014 4064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255: Bootstrap starting.
Aug 10 01:01:06 dist-test-slave-gp6n krb5kdc[3970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:01:06.329340 4064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255: Neither blocks nor log segments found. Creating new log.
I20260810 01:01:06.330804 4064 log.cc:824] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255: Log is configured to *not* fsync() on all Append() calls
I20260810 01:01:06.334422 4064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255: No bootstrap required, opened a new log
I20260810 01:01:06.335479 4067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:06.311179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47880 (local address 127.20.81.190:40789)
0810 01:01:06.311999 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:06.312016 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:06.312064 (+ 48us) server_negotiation.cc:407] Connection header received
0810 01:01:06.314453 (+ 2389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:06.314484 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:06.315021 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:06.315498 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:06.317045 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:06.319158 (+ 2113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:06.320631 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:06.321708 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:06.321927 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:06.326262 (+ 4335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:06.326273 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:06.326293 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:06.326343 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:06.330522 (+ 4179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:06.331332 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:06.331344 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:06.331361 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:06.331644 (+ 283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:06.332078 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:06.332085 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:06.332089 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:06.332843 (+ 754us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:06.333157 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:06.333595 (+ 438us) server_negotiation.cc:299] Negotiation successful
0810 01:01:06.334061 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":334,"threads_started":1}
I20260810 01:01:06.339764 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 01:01:06.344784 4064 raft_consensus.cc:348] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85a013f4e3448ddb042201ee9325255" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40789 } }
I20260810 01:01:06.345129 4064 raft_consensus.cc:374] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:01:06.345207 4064 raft_consensus.cc:729] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e85a013f4e3448ddb042201ee9325255, State: Initialized, Role: FOLLOWER
I20260810 01:01:06.345758 4064 consensus_queue.cc:260] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85a013f4e3448ddb042201ee9325255" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40789 } }
I20260810 01:01:06.345958 4064 raft_consensus.cc:388] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:01:06.346066 4064 raft_consensus.cc:482] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:01:06.346186 4064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:01:06.348320 4064 raft_consensus.cc:504] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85a013f4e3448ddb042201ee9325255" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40789 } }
I20260810 01:01:06.348762 4064 leader_election.cc:302] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e85a013f4e3448ddb042201ee9325255; no voters:
I20260810 01:01:06.349301 4064 leader_election.cc:288] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:01:06.349472 4069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:01:06.350189 4069 raft_consensus.cc:686] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255 [term 1 LEADER]: Becoming Leader. State: Replica: e85a013f4e3448ddb042201ee9325255, State: Running, Role: LEADER
I20260810 01:01:06.350715 4069 consensus_queue.cc:237] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85a013f4e3448ddb042201ee9325255" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40789 } }
I20260810 01:01:06.351444 4064 sys_catalog.cc:564] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:01:06.354024 4071 sys_catalog.cc:455] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e85a013f4e3448ddb042201ee9325255. Latest consensus state: current_term: 1 leader_uuid: "e85a013f4e3448ddb042201ee9325255" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85a013f4e3448ddb042201ee9325255" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40789 } } }
I20260810 01:01:06.354468 4071 sys_catalog.cc:458] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255 [sys.catalog]: This master's current role is: LEADER
I20260810 01:01:06.354825 4070 sys_catalog.cc:455] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e85a013f4e3448ddb042201ee9325255" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85a013f4e3448ddb042201ee9325255" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40789 } } }
I20260810 01:01:06.355561 4070 sys_catalog.cc:458] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255 [sys.catalog]: This master's current role is: LEADER
I20260810 01:01:06.359596 4077 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:01:06.365231 4077 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:01:06.375366 4077 catalog_manager.cc:1347] Generated new cluster ID: 6f7a28cd7b5d4d61bc9090a69aa452c3
I20260810 01:01:06.375510 4077 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:01:06.402598 4077 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:01:06.404035 4077 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:01:06.413254 4077 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e85a013f4e3448ddb042201ee9325255: Generated new TSK 0
I20260810 01:01:06.414462 4077 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:01:06.454617 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 3984
I20260810 01:01:06.455952 4059 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:06.553018 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 3984
2026-08-10T01:01:06Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2251 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:01:06 dist-test-slave-gp6n krb5kdc[4095](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:01:06 dist-test-slave-gp6n krb5kdc[4095](info): set up 2 sockets
Aug 10 01:01:06 dist-test-slave-gp6n krb5kdc[4095](info): commencing operation
krb5kdc: starting...
W20260810 01:01:07.766635 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:01:07 dist-test-slave-gp6n krb5kdc[4095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:01:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:01:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 01:01:08.067281 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:38513
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:38513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:01:08.417701 4111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:08.417984 4111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:08.418154 4111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:08.425518 4111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:01:08.425623 4111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:08.425743 4111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:08.425853 4111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:01:08.425941 4111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:01:08.435880 4111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:38513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:38513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:08.437232 4111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:08.438700 4111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:01:08.451267 4116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:08.451231 4117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:08.451313 4119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:08.452333 4111 server_base.cc:1034] running on GCE node
I20260810 01:01:08.453215 4111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:08.454887 4111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:08.456130 4111 hybrid_clock.cc:648] HybridClock initialized: now 1786323668456087 us; error 58 us; skew 500 ppm
I20260810 01:01:08.456513 4111 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:08 dist-test-slave-gp6n krb5kdc[4095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323668, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:08.459959 4111 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 01:01:08.461429 4111 webserver.cc:460] Webserver started at http://127.20.81.190:43005/ using document root <none> and password file <none>
I20260810 01:01:08.462138 4111 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:08.462241 4111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:08.462581 4111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:01:08.465654 4111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "5d1ff61743224f77bab36de848a90268"
format_stamp: "Formatted at 2026-08-10 01:01:08 on dist-test-slave-gp6n"
server_key: "a35b53fdf650ba0968fa4e9706fd7ec5"
server_key_iv: "23beb2b95b05319a3be4b96847b4e11a"
server_key_version: "encryptionkey@0"
I20260810 01:01:08.466368 4111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "5d1ff61743224f77bab36de848a90268"
format_stamp: "Formatted at 2026-08-10 01:01:08 on dist-test-slave-gp6n"
server_key: "a35b53fdf650ba0968fa4e9706fd7ec5"
server_key_iv: "23beb2b95b05319a3be4b96847b4e11a"
server_key_version: "encryptionkey@0"
I20260810 01:01:08.471671 4111 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 01:01:08.475090 4126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:08.476413 4111 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 01:01:08.476627 4111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "5d1ff61743224f77bab36de848a90268"
format_stamp: "Formatted at 2026-08-10 01:01:08 on dist-test-slave-gp6n"
server_key: "a35b53fdf650ba0968fa4e9706fd7ec5"
server_key_iv: "23beb2b95b05319a3be4b96847b4e11a"
server_key_version: "encryptionkey@0"
I20260810 01:01:08.476831 4111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:08.508564 4111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:08.513301 4111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:08.513634 4111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:08.532879 4111 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:38513
I20260810 01:01:08.532914 4185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:38513 every 8 connection(s)
I20260810 01:01:08.534337 4111 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:01:08.535883 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 4111
I20260810 01:01:08.536120 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:01:08.537020 20806 external_mini_cluster.cc:1428] Setting key 897179d7dc7a902342d064bd2cd754ef
I20260810 01:01:08.539269 4187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 01:01:08 dist-test-slave-gp6n krb5kdc[4095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:01:08.553439 4187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268: Bootstrap starting.
I20260810 01:01:08.556810 4187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268: Neither blocks nor log segments found. Creating new log.
I20260810 01:01:08.558152 4187 log.cc:824] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268: Log is configured to *not* fsync() on all Append() calls
I20260810 01:01:08.561293 4190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:08.539230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51064 (local address 127.20.81.190:38513)
0810 01:01:08.540109 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:08.540121 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:08.540163 (+ 42us) server_negotiation.cc:407] Connection header received
0810 01:01:08.541713 (+ 1550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:08.541734 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:08.542158 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:08.542592 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:08.544399 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:08.545892 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:08.547761 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:08.548413 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:08.548553 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:08.553807 (+ 5254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:08.553814 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:08.553829 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:08.553865 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:08.557026 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:08.557844 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:08.557850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:08.557857 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:08.558032 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:08.558499 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:08.558503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:08.558505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:08.559238 (+ 733us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:08.559610 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:08.559917 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 01:01:08.560264 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":401,"threads_started":1}
I20260810 01:01:08.561422 4187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268: No bootstrap required, opened a new log
I20260810 01:01:08.569123 4187 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1ff61743224f77bab36de848a90268" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38513 } }
I20260810 01:01:08.569530 4187 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:01:08.569653 4187 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d1ff61743224f77bab36de848a90268, State: Initialized, Role: FOLLOWER
I20260810 01:01:08.570206 4187 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1ff61743224f77bab36de848a90268" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38513 } }
I20260810 01:01:08.570506 4187 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:01:08.570667 4187 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:01:08.570847 4187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:01:08.572993 4187 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1ff61743224f77bab36de848a90268" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38513 } }
I20260810 01:01:08.573472 4187 leader_election.cc:302] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d1ff61743224f77bab36de848a90268; no voters:
I20260810 01:01:08.574080 4187 leader_election.cc:288] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:01:08.574213 4192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:01:08.574975 4192 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 1 LEADER]: Becoming Leader. State: Replica: 5d1ff61743224f77bab36de848a90268, State: Running, Role: LEADER
I20260810 01:01:08.575528 4192 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1ff61743224f77bab36de848a90268" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38513 } }
I20260810 01:01:08.576192 4187 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:01:08.578293 4194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d1ff61743224f77bab36de848a90268. Latest consensus state: current_term: 1 leader_uuid: "5d1ff61743224f77bab36de848a90268" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1ff61743224f77bab36de848a90268" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38513 } } }
I20260810 01:01:08.578758 4194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [sys.catalog]: This master's current role is: LEADER
I20260810 01:01:08.580682 4200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:01:08.580525 4193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d1ff61743224f77bab36de848a90268" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1ff61743224f77bab36de848a90268" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38513 } } }
I20260810 01:01:08.580981 4193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [sys.catalog]: This master's current role is: LEADER
I20260810 01:01:08.585826 4200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:01:08.594396 4200 catalog_manager.cc:1347] Generated new cluster ID: 1fb0ad0d74ee445fbd32233456b2f9c2
I20260810 01:01:08.594556 4200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:01:08.618247 4200 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:01:08.619638 4200 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:01:08.641794 4200 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268: Generated new TSK 0
I20260810 01:01:08.642782 4200 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.81.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
I20260810 01:01:08.731284 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:38513
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:01:09.076287 4214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:09.076578 4214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:09.076731 4214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:09.084705 4214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:09.084821 4214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:09.084939 4214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 01:01:09.094928 4214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:38513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:09.096395 4214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:09.097750 4214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:01:09.111406 4220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:09.111650 4221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:09.112155 4223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:09.112295 4214 server_base.cc:1034] running on GCE node
I20260810 01:01:09.112999 4214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:09.113996 4214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:09.115235 4214 hybrid_clock.cc:648] HybridClock initialized: now 1786323669115199 us; error 46 us; skew 500 ppm
I20260810 01:01:09.115615 4214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:09 dist-test-slave-gp6n krb5kdc[4095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323669, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:09.118964 4214 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 01:01:09.120414 4214 webserver.cc:460] Webserver started at http://127.20.81.129:46621/ using document root <none> and password file <none>
I20260810 01:01:09.121157 4214 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:09.121317 4214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:09.121675 4214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:01:09.124893 4214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "85cb24da8f7941869b014c3b90e3b3da"
format_stamp: "Formatted at 2026-08-10 01:01:09 on dist-test-slave-gp6n"
server_key: "5660f6f65c146709066d31c050fa1925"
server_key_iv: "e0ec9e2db86e01d66dc94fed693fce33"
server_key_version: "encryptionkey@0"
I20260810 01:01:09.125736 4214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "85cb24da8f7941869b014c3b90e3b3da"
format_stamp: "Formatted at 2026-08-10 01:01:09 on dist-test-slave-gp6n"
server_key: "5660f6f65c146709066d31c050fa1925"
server_key_iv: "e0ec9e2db86e01d66dc94fed693fce33"
server_key_version: "encryptionkey@0"
I20260810 01:01:09.131238 4214 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 01:01:09.134806 4232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:09.136158 4214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 01:01:09.136349 4214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "85cb24da8f7941869b014c3b90e3b3da"
format_stamp: "Formatted at 2026-08-10 01:01:09 on dist-test-slave-gp6n"
server_key: "5660f6f65c146709066d31c050fa1925"
server_key_iv: "e0ec9e2db86e01d66dc94fed693fce33"
server_key_version: "encryptionkey@0"
I20260810 01:01:09.136579 4214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:09.151777 4214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:09.155925 4214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:09.156243 4214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:09.157238 4214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:01:09.158705 4214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:01:09.158815 4214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:09.158967 4214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:01:09.159049 4214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:09.193745 4214 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:46663
I20260810 01:01:09.193768 4358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:46663 every 8 connection(s)
I20260810 01:01:09.195154 4214 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 01:01:09.200150 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 4214
I20260810 01:01:09.200384 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 01:01:09.201213 20806 external_mini_cluster.cc:1428] Setting key 7c4adcdc763e4d232c471bea7ad0330f
Aug 10 01:01:09 dist-test-slave-gp6n krb5kdc[4095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323669, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:01:09.220633 4361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:09.197743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:49103 (local address 127.20.81.190:38513)
0810 01:01:09.198200 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:09.198205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:09.199144 (+ 939us) server_negotiation.cc:407] Connection header received
0810 01:01:09.201744 (+ 2600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:09.201749 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:09.201816 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:09.201968 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:09.203840 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:09.204541 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:09.207025 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:09.207428 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:09.207527 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:09.212373 (+ 4846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:09.212382 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:09.212389 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:09.212433 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:09.216799 (+ 4366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:09.217825 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:09.217834 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:09.217841 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:09.218002 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:09.218539 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:09.218547 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:09.218553 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:09.218949 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:09.219109 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:09.219831 (+ 722us) server_negotiation.cc:299] Negotiation successful
0810 01:01:09.220118 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":238,"threads_started":1}
I20260810 01:01:09.221344 4362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:09.198279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.129:49103)
0810 01:01:09.198996 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:09.199034 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:09.201413 (+ 2379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:09.202215 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:09.202229 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:09.202841 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:01:09.203655 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:09.203684 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:09.204712 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:09.204720 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:09.206867 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:09.206885 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:09.207661 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:09.207670 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:09.208015 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:09.208763 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:01:09.208797 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:01:09.212067 (+ 3270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:01:09.216968 (+ 4901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:09.216978 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:09.216999 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:09.217667 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:09.218118 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:09.218122 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:09.218124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:09.218350 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:09.219077 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:01:09.219083 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:01:09.219424 (+ 341us) client_negotiation.cc:769] Sending connection context
0810 01:01:09.219655 (+ 231us) client_negotiation.cc:240] Negotiation successful
0810 01:01:09.220081 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":297,"threads_started":1}
I20260810 01:01:09.223083 4359 heartbeater.cc:347] Connected to a master server at 127.20.81.190:38513
I20260810 01:01:09.223418 4359 heartbeater.cc:464] Registering TS with master...
I20260810 01:01:09.224188 4359 heartbeater.cc:511] Master 127.20.81.190:38513 requested a full tablet report, sending...
I20260810 01:01:09.226261 4145 ts_manager.cc:194] Registered new tserver with Master: 85cb24da8f7941869b014c3b90e3b3da (127.20.81.129:46663)
I20260810 01:01:09.227762 4145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:49103
WARNING: no policy specified for kudu/127.20.81.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
I20260810 01:01:09.270479 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:38513
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:01:09.621925 4365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:09.622203 4365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:09.622299 4365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:09.629732 4365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:09.629844 4365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:09.629969 4365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 01:01:09.640101 4365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:38513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:09.641515 4365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:09.642976 4365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:01:09.657253 4371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:09.657188 4370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:09.657192 4373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:09.658350 4365 server_base.cc:1034] running on GCE node
I20260810 01:01:09.659044 4365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:09.660094 4365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:09.661337 4365 hybrid_clock.cc:648] HybridClock initialized: now 1786323669661296 us; error 54 us; skew 500 ppm
I20260810 01:01:09.661671 4365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:09 dist-test-slave-gp6n krb5kdc[4095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323669, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:09.665117 4365 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 01:01:09.666611 4365 webserver.cc:460] Webserver started at http://127.20.81.130:39021/ using document root <none> and password file <none>
I20260810 01:01:09.667316 4365 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:09.667490 4365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:09.667829 4365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:01:09.670885 4365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "825440956e2e4d4aa81efda8a3262648"
format_stamp: "Formatted at 2026-08-10 01:01:09 on dist-test-slave-gp6n"
server_key: "ac6f1a60c97ca6f7ae4fb0a3be792c59"
server_key_iv: "c88e07166452f6dd82f9428cfbbe21b0"
server_key_version: "encryptionkey@0"
I20260810 01:01:09.671686 4365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "825440956e2e4d4aa81efda8a3262648"
format_stamp: "Formatted at 2026-08-10 01:01:09 on dist-test-slave-gp6n"
server_key: "ac6f1a60c97ca6f7ae4fb0a3be792c59"
server_key_iv: "c88e07166452f6dd82f9428cfbbe21b0"
server_key_version: "encryptionkey@0"
I20260810 01:01:09.676847 4365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 01:01:09.680398 4380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:09.681782 4365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 01:01:09.681962 4365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "825440956e2e4d4aa81efda8a3262648"
format_stamp: "Formatted at 2026-08-10 01:01:09 on dist-test-slave-gp6n"
server_key: "ac6f1a60c97ca6f7ae4fb0a3be792c59"
server_key_iv: "c88e07166452f6dd82f9428cfbbe21b0"
server_key_version: "encryptionkey@0"
I20260810 01:01:09.682165 4365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:09.698493 4365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:09.703125 4365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:09.703444 4365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:09.704440 4365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:01:09.705847 4365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:01:09.705966 4365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:09.706139 4365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:01:09.706215 4365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:09.740363 4365 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:35483
I20260810 01:01:09.740422 4509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:35483 every 8 connection(s)
I20260810 01:01:09.741838 4365 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 01:01:09.751509 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 4365
I20260810 01:01:09.751714 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 01:01:09.752200 20806 external_mini_cluster.cc:1428] Setting key 8645304ae3568cdd84659a8994530673
Aug 10 01:01:09 dist-test-slave-gp6n krb5kdc[4095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323669, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:01:09.764415 4512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:09.744526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:41113 (local address 127.20.81.190:38513)
0810 01:01:09.745023 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:09.745029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:09.745892 (+ 863us) server_negotiation.cc:407] Connection header received
0810 01:01:09.747878 (+ 1986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:09.747884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:09.747985 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:09.748133 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:09.749691 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:09.750493 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:09.752371 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:09.752970 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:09.753123 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:09.757255 (+ 4132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:09.757264 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:09.757271 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:09.757318 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:09.761051 (+ 3733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:09.761799 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:09.761807 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:09.761814 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:09.761985 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:09.762421 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:09.762429 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:09.762433 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:09.762814 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:09.762983 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:09.763647 (+ 664us) server_negotiation.cc:299] Negotiation successful
0810 01:01:09.763908 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":266,"threads_started":1}
I20260810 01:01:09.765210 4513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:09.744996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.130:41113)
0810 01:01:09.745718 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:09.745759 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:09.747586 (+ 1827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:09.748386 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:09.748395 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:09.748916 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:01:09.749518 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:09.749535 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:09.750708 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:09.750712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:09.752178 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:09.752191 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:09.753243 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:09.753250 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:09.753502 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:09.754205 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:01:09.754268 (+ 63us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:01:09.757007 (+ 2739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:01:09.761199 (+ 4192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:09.761206 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:09.761218 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:09.761668 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:09.762099 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:09.762102 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:09.762105 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:09.762289 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:09.762941 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:01:09.762947 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:01:09.763266 (+ 319us) client_negotiation.cc:769] Sending connection context
0810 01:01:09.763511 (+ 245us) client_negotiation.cc:240] Negotiation successful
0810 01:01:09.763991 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":283,"threads_started":1}
I20260810 01:01:09.766801 4510 heartbeater.cc:347] Connected to a master server at 127.20.81.190:38513
I20260810 01:01:09.767129 4510 heartbeater.cc:464] Registering TS with master...
I20260810 01:01:09.767895 4510 heartbeater.cc:511] Master 127.20.81.190:38513 requested a full tablet report, sending...
I20260810 01:01:09.769608 4145 ts_manager.cc:194] Registered new tserver with Master: 825440956e2e4d4aa81efda8a3262648 (127.20.81.130:35483)
I20260810 01:01:09.770496 4145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:41113
WARNING: no policy specified for kudu/127.20.81.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
I20260810 01:01:09.818641 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:38513
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:01:10.163986 4516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:10.164350 4516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:10.164529 4516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:10.172003 4516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:10.172241 4516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:10.172449 4516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 01:01:10.182595 4516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:38513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:10.184146 4516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:10.185513 4516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:01:10.198828 4523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:10.199167 4524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:10.199193 4526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:10.200219 4516 server_base.cc:1034] running on GCE node
I20260810 01:01:10.200824 4516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:10.201717 4516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:10.202966 4516 hybrid_clock.cc:648] HybridClock initialized: now 1786323670202924 us; error 36 us; skew 500 ppm
I20260810 01:01:10.203331 4516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:10 dist-test-slave-gp6n krb5kdc[4095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323670, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:10.206583 4516 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 01:01:10.207945 4516 webserver.cc:460] Webserver started at http://127.20.81.131:34591/ using document root <none> and password file <none>
I20260810 01:01:10.208626 4516 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:10.208781 4516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:10.209077 4516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:01:10.211992 4516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "f4fff50a490a4e29bfa942a2bf11ff0e"
format_stamp: "Formatted at 2026-08-10 01:01:10 on dist-test-slave-gp6n"
server_key: "79b4b035629a7d2ce6a4f116b22f93f9"
server_key_iv: "caba9d50c71fef6a89bd5d5015d1d5b6"
server_key_version: "encryptionkey@0"
I20260810 01:01:10.212771 4516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "f4fff50a490a4e29bfa942a2bf11ff0e"
format_stamp: "Formatted at 2026-08-10 01:01:10 on dist-test-slave-gp6n"
server_key: "79b4b035629a7d2ce6a4f116b22f93f9"
server_key_iv: "caba9d50c71fef6a89bd5d5015d1d5b6"
server_key_version: "encryptionkey@0"
I20260810 01:01:10.219152 4516 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 01:01:10.224761 4533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:10.226279 4516 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.003s
I20260810 01:01:10.226470 4516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "f4fff50a490a4e29bfa942a2bf11ff0e"
format_stamp: "Formatted at 2026-08-10 01:01:10 on dist-test-slave-gp6n"
server_key: "79b4b035629a7d2ce6a4f116b22f93f9"
server_key_iv: "caba9d50c71fef6a89bd5d5015d1d5b6"
server_key_version: "encryptionkey@0"
I20260810 01:01:10.226663 4516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:10.230711 4359 heartbeater.cc:503] Master 127.20.81.190:38513 was elected leader, sending a full tablet report...
I20260810 01:01:10.256523 4516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:10.260756 4516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:10.261013 4516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:10.261982 4516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:01:10.263363 4516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:01:10.263438 4516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:10.263587 4516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:01:10.263641 4516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:10.299304 4516 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:39567
I20260810 01:01:10.299356 4658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:39567 every 8 connection(s)
I20260810 01:01:10.300648 4516 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 01:01:10.307237 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 4516
I20260810 01:01:10.307475 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 01:01:10.307953 20806 external_mini_cluster.cc:1428] Setting key 539e9a1f48b05706cc8edb3c9805b9d3
Aug 10 01:01:10 dist-test-slave-gp6n krb5kdc[4095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323670, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:01:10.321700 4662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:10.303328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:60327 (local address 127.20.81.190:38513)
0810 01:01:10.303847 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:10.303854 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:10.304838 (+ 984us) server_negotiation.cc:407] Connection header received
0810 01:01:10.306733 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:10.306738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:10.306846 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:10.306991 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:10.308824 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.309884 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.311577 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.312024 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.312168 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:10.315891 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:10.315897 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:10.315900 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:10.315941 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:10.318548 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:10.319287 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:10.319292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:10.319296 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:10.319425 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:10.319880 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:10.319885 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:10.319890 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:10.320175 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:10.320340 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:10.321069 (+ 729us) server_negotiation.cc:299] Negotiation successful
0810 01:01:10.321299 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":227,"threads_started":1}
I20260810 01:01:10.322877 4664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:10.303941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.131:60327)
0810 01:01:10.304662 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:10.304703 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:10.306392 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:10.307192 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:10.307200 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:10.307958 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:01:10.308606 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:10.308651 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.310051 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.310056 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:10.311407 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:10.311419 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.312265 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.312273 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:10.312519 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:10.313266 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:01:10.313307 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:01:10.315664 (+ 2357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:01:10.318689 (+ 3025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:10.318698 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:10.318715 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:10.319151 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:10.319546 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:10.319551 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:10.319555 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:10.319751 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:10.320310 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:01:10.320318 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:01:10.320682 (+ 364us) client_negotiation.cc:769] Sending connection context
0810 01:01:10.320951 (+ 269us) client_negotiation.cc:240] Negotiation successful
0810 01:01:10.321511 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":280,"threads_started":1}
I20260810 01:01:10.324283 4659 heartbeater.cc:347] Connected to a master server at 127.20.81.190:38513
I20260810 01:01:10.324577 4659 heartbeater.cc:464] Registering TS with master...
I20260810 01:01:10.325280 4659 heartbeater.cc:511] Master 127.20.81.190:38513 requested a full tablet report, sending...
I20260810 01:01:10.326721 4145 ts_manager.cc:194] Registered new tserver with Master: f4fff50a490a4e29bfa942a2bf11ff0e (127.20.81.131:39567)
I20260810 01:01:10.327566 4145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:60327
I20260810 01:01:10.336288 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 01:01:10.353938 4662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:10.341138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51072 (local address 127.20.81.190:38513)
0810 01:01:10.341356 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:10.341363 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:10.341823 (+ 460us) server_negotiation.cc:407] Connection header received
0810 01:01:10.341956 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:10.341959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:10.342035 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:10.342119 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:10.343590 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.344347 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.345774 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.346184 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.346277 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:10.348773 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:10.348778 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:10.348781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:10.348812 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:10.351460 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:10.352149 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:10.352153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:10.352155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:10.352280 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:10.352624 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:10.352627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:10.352629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:10.352945 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:10.353148 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:10.353439 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 01:01:10.353609 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 01:01:10.357854 4145 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51072:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 01:01:10.364794 4145 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 01:01:10.389308 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:10.378195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42914 (local address 127.20.81.129:46663)
0810 01:01:10.379137 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:10.379143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:10.380392 (+ 1249us) server_negotiation.cc:407] Connection header received
0810 01:01:10.380751 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:10.380759 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:10.381045 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:10.381253 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:01:10.382117 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.383826 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.386955 (+ 3129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.388058 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.388207 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:10.388445 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:10.388783 (+ 338us) server_negotiation.cc:299] Negotiation successful
0810 01:01:10.388954 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":807,"thread_start_us":673,"threads_started":1}
I20260810 01:01:10.389308 4674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:10.378005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:46663 (local address 127.0.0.1:42914)
0810 01:01:10.380219 (+ 2214us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:10.380238 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:10.380550 (+ 312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:10.381530 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:10.381535 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:10.381564 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:01:10.381898 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:10.381906 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.384237 (+ 2331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.384242 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:10.386799 (+ 2557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:10.386825 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.388310 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.388321 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:10.388535 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:10.388597 (+ 62us) client_negotiation.cc:769] Sending connection context
0810 01:01:10.388738 (+ 141us) client_negotiation.cc:240] Negotiation successful
0810 01:01:10.388921 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2043,"thread_start_us":1250,"threads_started":1}
I20260810 01:01:10.390343 4675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:10.377356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58218 (local address 127.20.81.130:35483)
0810 01:01:10.378709 (+ 1353us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:10.378717 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:10.379717 (+ 1000us) server_negotiation.cc:407] Connection header received
0810 01:01:10.380143 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:10.380152 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:10.382569 (+ 2417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:10.382808 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:01:10.383666 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.385451 (+ 1785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.387794 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.389238 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.389412 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:10.389618 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:10.389857 (+ 239us) server_negotiation.cc:299] Negotiation successful
0810 01:01:10.390014 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1196,"thread_start_us":1088,"threads_started":1}
I20260810 01:01:10.390365 4676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:10.377226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:35483 (local address 127.0.0.1:58218)
0810 01:01:10.379558 (+ 2332us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:10.379597 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:10.379825 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:10.383060 (+ 3235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:10.383064 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:10.383092 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:01:10.383480 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:10.383499 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.385594 (+ 2095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.385599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:10.387631 (+ 2032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:10.387643 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.389471 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.389478 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:10.389658 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:10.389696 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 01:01:10.389807 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 01:01:10.389933 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2137,"mutex_wait_us":43,"thread_start_us":2641,"threads_started":1}
I20260810 01:01:10.393488 4287 tablet_service.cc:1467] Processing CreateTablet for tablet 32962e08eeb24f678200a1e2b53b3bfa (DEFAULT_TABLE table=test-table [id=caad1d3a7b9f443a90793b2bb9d557d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:01:10.394325 4678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:10.380215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:39567 (local address 127.0.0.1:50556)
0810 01:01:10.382504 (+ 2289us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:10.382522 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:10.382640 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:10.383847 (+ 1207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:10.383852 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:10.383880 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:01:10.384280 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:10.384294 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.386138 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.386145 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:10.391289 (+ 5144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:10.391326 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.393494 (+ 2168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.393500 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:10.393674 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:10.393705 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 01:01:10.393798 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 01:01:10.393922 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2154,"mutex_wait_us":16,"thread_start_us":2010,"threads_started":1}
I20260810 01:01:10.394155 4434 tablet_service.cc:1467] Processing CreateTablet for tablet 32962e08eeb24f678200a1e2b53b3bfa (DEFAULT_TABLE table=test-table [id=caad1d3a7b9f443a90793b2bb9d557d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:01:10.395066 4679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:10.380628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50556 (local address 127.20.81.131:39567)
0810 01:01:10.383213 (+ 2585us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:10.383223 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:10.383259 (+ 36us) server_negotiation.cc:407] Connection header received
0810 01:01:10.383349 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:10.383355 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:10.383552 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:10.383723 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:01:10.384515 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.386000 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.391531 (+ 5531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.392975 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.393116 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:10.393341 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:10.394549 (+ 1208us) server_negotiation.cc:299] Negotiation successful
0810 01:01:10.394742 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2403,"thread_start_us":2233,"threads_started":1}
I20260810 01:01:10.395586 4434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32962e08eeb24f678200a1e2b53b3bfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:01:10.395771 4287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32962e08eeb24f678200a1e2b53b3bfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:01:10.398469 4589 tablet_service.cc:1467] Processing CreateTablet for tablet 32962e08eeb24f678200a1e2b53b3bfa (DEFAULT_TABLE table=test-table [id=caad1d3a7b9f443a90793b2bb9d557d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:01:10.399827 4589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32962e08eeb24f678200a1e2b53b3bfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:01:10.408844 4680 tablet_bootstrap.cc:492] T 32962e08eeb24f678200a1e2b53b3bfa P 825440956e2e4d4aa81efda8a3262648: Bootstrap starting.
I20260810 01:01:10.410912 4681 tablet_bootstrap.cc:492] T 32962e08eeb24f678200a1e2b53b3bfa P 85cb24da8f7941869b014c3b90e3b3da: Bootstrap starting.
I20260810 01:01:10.412928 4680 tablet_bootstrap.cc:654] T 32962e08eeb24f678200a1e2b53b3bfa P 825440956e2e4d4aa81efda8a3262648: Neither blocks nor log segments found. Creating new log.
I20260810 01:01:10.413926 4682 tablet_bootstrap.cc:492] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e: Bootstrap starting.
I20260810 01:01:10.414276 4680 log.cc:824] T 32962e08eeb24f678200a1e2b53b3bfa P 825440956e2e4d4aa81efda8a3262648: Log is configured to *not* fsync() on all Append() calls
I20260810 01:01:10.414770 4681 tablet_bootstrap.cc:654] T 32962e08eeb24f678200a1e2b53b3bfa P 85cb24da8f7941869b014c3b90e3b3da: Neither blocks nor log segments found. Creating new log.
I20260810 01:01:10.416339 4681 log.cc:824] T 32962e08eeb24f678200a1e2b53b3bfa P 85cb24da8f7941869b014c3b90e3b3da: Log is configured to *not* fsync() on all Append() calls
I20260810 01:01:10.417275 4682 tablet_bootstrap.cc:654] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e: Neither blocks nor log segments found. Creating new log.
I20260810 01:01:10.418215 4682 log.cc:824] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e: Log is configured to *not* fsync() on all Append() calls
I20260810 01:01:10.421021 4682 tablet_bootstrap.cc:492] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e: No bootstrap required, opened a new log
I20260810 01:01:10.421293 4682 ts_tablet_manager.cc:1397] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260810 01:01:10.421307 4680 tablet_bootstrap.cc:492] T 32962e08eeb24f678200a1e2b53b3bfa P 825440956e2e4d4aa81efda8a3262648: No bootstrap required, opened a new log
I20260810 01:01:10.421391 4681 tablet_bootstrap.cc:492] T 32962e08eeb24f678200a1e2b53b3bfa P 85cb24da8f7941869b014c3b90e3b3da: No bootstrap required, opened a new log
I20260810 01:01:10.421583 4680 ts_tablet_manager.cc:1397] T 32962e08eeb24f678200a1e2b53b3bfa P 825440956e2e4d4aa81efda8a3262648: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260810 01:01:10.421633 4681 ts_tablet_manager.cc:1397] T 32962e08eeb24f678200a1e2b53b3bfa P 85cb24da8f7941869b014c3b90e3b3da: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260810 01:01:10.428736 4682 raft_consensus.cc:348] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825440956e2e4d4aa81efda8a3262648" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 35483 } } peers { permanent_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39567 } } peers { permanent_uuid: "85cb24da8f7941869b014c3b90e3b3da" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 46663 } }
I20260810 01:01:10.429113 4682 raft_consensus.cc:374] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:01:10.429220 4682 raft_consensus.cc:729] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4fff50a490a4e29bfa942a2bf11ff0e, State: Initialized, Role: FOLLOWER
I20260810 01:01:10.429912 4682 consensus_queue.cc:260] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825440956e2e4d4aa81efda8a3262648" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 35483 } } peers { permanent_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39567 } } peers { permanent_uuid: "85cb24da8f7941869b014c3b90e3b3da" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 46663 } }
I20260810 01:01:10.431123 4659 heartbeater.cc:503] Master 127.20.81.190:38513 was elected leader, sending a full tablet report...
I20260810 01:01:10.431921 4682 ts_tablet_manager.cc:1428] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e: Time spent starting tablet: real 0.010s user 0.012s sys 0.000s
I20260810 01:01:10.432457 4681 raft_consensus.cc:348] T 32962e08eeb24f678200a1e2b53b3bfa P 85cb24da8f7941869b014c3b90e3b3da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825440956e2e4d4aa81efda8a3262648" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 35483 } } peers { permanent_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39567 } } peers { permanent_uuid: "85cb24da8f7941869b014c3b90e3b3da" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 46663 } }
I20260810 01:01:10.432456 4680 raft_consensus.cc:348] T 32962e08eeb24f678200a1e2b53b3bfa P 825440956e2e4d4aa81efda8a3262648 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825440956e2e4d4aa81efda8a3262648" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 35483 } } peers { permanent_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39567 } } peers { permanent_uuid: "85cb24da8f7941869b014c3b90e3b3da" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 46663 } }
I20260810 01:01:10.432906 4681 raft_consensus.cc:374] T 32962e08eeb24f678200a1e2b53b3bfa P 85cb24da8f7941869b014c3b90e3b3da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:01:10.432905 4680 raft_consensus.cc:374] T 32962e08eeb24f678200a1e2b53b3bfa P 825440956e2e4d4aa81efda8a3262648 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:01:10.433012 4680 raft_consensus.cc:729] T 32962e08eeb24f678200a1e2b53b3bfa P 825440956e2e4d4aa81efda8a3262648 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825440956e2e4d4aa81efda8a3262648, State: Initialized, Role: FOLLOWER
I20260810 01:01:10.433013 4681 raft_consensus.cc:729] T 32962e08eeb24f678200a1e2b53b3bfa P 85cb24da8f7941869b014c3b90e3b3da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85cb24da8f7941869b014c3b90e3b3da, State: Initialized, Role: FOLLOWER
I20260810 01:01:10.433622 4681 consensus_queue.cc:260] T 32962e08eeb24f678200a1e2b53b3bfa P 85cb24da8f7941869b014c3b90e3b3da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825440956e2e4d4aa81efda8a3262648" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 35483 } } peers { permanent_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39567 } } peers { permanent_uuid: "85cb24da8f7941869b014c3b90e3b3da" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 46663 } }
I20260810 01:01:10.433620 4680 consensus_queue.cc:260] T 32962e08eeb24f678200a1e2b53b3bfa P 825440956e2e4d4aa81efda8a3262648 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825440956e2e4d4aa81efda8a3262648" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 35483 } } peers { permanent_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39567 } } peers { permanent_uuid: "85cb24da8f7941869b014c3b90e3b3da" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 46663 } }
I20260810 01:01:10.435495 4680 ts_tablet_manager.cc:1428] T 32962e08eeb24f678200a1e2b53b3bfa P 825440956e2e4d4aa81efda8a3262648: Time spent starting tablet: real 0.014s user 0.012s sys 0.003s
I20260810 01:01:10.435628 4681 ts_tablet_manager.cc:1428] T 32962e08eeb24f678200a1e2b53b3bfa P 85cb24da8f7941869b014c3b90e3b3da: Time spent starting tablet: real 0.014s user 0.008s sys 0.007s
I20260810 01:01:10.436669 4510 heartbeater.cc:503] Master 127.20.81.190:38513 was elected leader, sending a full tablet report...
I20260810 01:01:10.446924 4686 raft_consensus.cc:482] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:01:10.447218 4686 raft_consensus.cc:504] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825440956e2e4d4aa81efda8a3262648" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 35483 } } peers { permanent_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39567 } } peers { permanent_uuid: "85cb24da8f7941869b014c3b90e3b3da" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 46663 } }
W20260810 01:01:10.447531 4360 tablet.cc:2369] T 32962e08eeb24f678200a1e2b53b3bfa P 85cb24da8f7941869b014c3b90e3b3da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:01:10.449172 4686 leader_election.cc:288] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 825440956e2e4d4aa81efda8a3262648 (127.20.81.130:35483), 85cb24da8f7941869b014c3b90e3b3da (127.20.81.129:46663)
I20260810 01:01:10.456261 4664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:10.449458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:35483 (local address 127.20.81.131:50353)
0810 01:01:10.449852 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:10.449874 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:10.450012 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:10.450753 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:10.450757 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:10.450783 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:01:10.450952 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:10.450960 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.451954 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.451960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:10.453950 (+ 1990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:10.453965 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.455478 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.455481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:10.455683 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:10.455716 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 01:01:10.455810 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 01:01:10.455957 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243}
I20260810 01:01:10.456453 4675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:10.450178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:50353 (local address 127.20.81.130:35483)
0810 01:01:10.450394 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:10.450430 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:10.450450 (+ 20us) server_negotiation.cc:407] Connection header received
0810 01:01:10.450531 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:10.450536 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:10.450638 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:10.450732 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:01:10.451097 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.451810 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.454120 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.455331 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.455498 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:10.455581 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:10.456052 (+ 471us) server_negotiation.cc:299] Negotiation successful
0810 01:01:10.456189 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 01:01:10.456708 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:10.449725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:58559 (local address 127.20.81.129:46663)
0810 01:01:10.449915 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:10.449922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:10.450653 (+ 731us) server_negotiation.cc:407] Connection header received
0810 01:01:10.450816 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:10.450820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:10.450877 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:10.450987 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:01:10.451497 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.452202 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.454482 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.455705 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.455888 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:10.455958 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:10.456318 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 01:01:10.456446 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 01:01:10.457291 4689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:10.449744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:46663 (local address 127.20.81.131:58559)
0810 01:01:10.450534 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:10.450549 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:10.450666 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:10.451001 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:10.451006 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:10.451030 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:01:10.451339 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:10.451348 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.452381 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.452392 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:10.454295 (+ 1903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:10.454305 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.455915 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.455921 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:10.456120 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:10.456158 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 01:01:10.456865 (+ 707us) client_negotiation.cc:240] Negotiation successful
0810 01:01:10.457019 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":437,"threads_started":1}
I20260810 01:01:10.457741 4458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32962e08eeb24f678200a1e2b53b3bfa" candidate_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825440956e2e4d4aa81efda8a3262648" is_pre_election: true
I20260810 01:01:10.458320 4458 raft_consensus.cc:2454] T 32962e08eeb24f678200a1e2b53b3bfa P 825440956e2e4d4aa81efda8a3262648 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4fff50a490a4e29bfa942a2bf11ff0e in term 0.
I20260810 01:01:10.458367 4309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32962e08eeb24f678200a1e2b53b3bfa" candidate_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85cb24da8f7941869b014c3b90e3b3da" is_pre_election: true
I20260810 01:01:10.458992 4309 raft_consensus.cc:2454] T 32962e08eeb24f678200a1e2b53b3bfa P 85cb24da8f7941869b014c3b90e3b3da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4fff50a490a4e29bfa942a2bf11ff0e in term 0.
I20260810 01:01:10.459093 4536 leader_election.cc:302] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 825440956e2e4d4aa81efda8a3262648, f4fff50a490a4e29bfa942a2bf11ff0e; no voters:
I20260810 01:01:10.459528 4686 raft_consensus.cc:2781] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:01:10.459693 4686 raft_consensus.cc:482] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:01:10.459824 4686 raft_consensus.cc:3037] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:01:10.462110 4686 raft_consensus.cc:504] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825440956e2e4d4aa81efda8a3262648" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 35483 } } peers { permanent_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39567 } } peers { permanent_uuid: "85cb24da8f7941869b014c3b90e3b3da" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 46663 } }
I20260810 01:01:10.462832 4686 leader_election.cc:288] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e [CANDIDATE]: Term 1 election: Requested vote from peers 825440956e2e4d4aa81efda8a3262648 (127.20.81.130:35483), 85cb24da8f7941869b014c3b90e3b3da (127.20.81.129:46663)
I20260810 01:01:10.463251 4458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32962e08eeb24f678200a1e2b53b3bfa" candidate_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825440956e2e4d4aa81efda8a3262648"
I20260810 01:01:10.463287 4309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32962e08eeb24f678200a1e2b53b3bfa" candidate_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85cb24da8f7941869b014c3b90e3b3da"
I20260810 01:01:10.463454 4458 raft_consensus.cc:3037] T 32962e08eeb24f678200a1e2b53b3bfa P 825440956e2e4d4aa81efda8a3262648 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:01:10.463467 4309 raft_consensus.cc:3037] T 32962e08eeb24f678200a1e2b53b3bfa P 85cb24da8f7941869b014c3b90e3b3da [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:01:10.465770 4458 raft_consensus.cc:2454] T 32962e08eeb24f678200a1e2b53b3bfa P 825440956e2e4d4aa81efda8a3262648 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4fff50a490a4e29bfa942a2bf11ff0e in term 1.
I20260810 01:01:10.465770 4309 raft_consensus.cc:2454] T 32962e08eeb24f678200a1e2b53b3bfa P 85cb24da8f7941869b014c3b90e3b3da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4fff50a490a4e29bfa942a2bf11ff0e in term 1.
I20260810 01:01:10.466172 4536 leader_election.cc:302] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 825440956e2e4d4aa81efda8a3262648, f4fff50a490a4e29bfa942a2bf11ff0e; no voters:
I20260810 01:01:10.466563 4686 raft_consensus.cc:2781] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:01:10.467087 4686 raft_consensus.cc:686] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e [term 1 LEADER]: Becoming Leader. State: Replica: f4fff50a490a4e29bfa942a2bf11ff0e, State: Running, Role: LEADER
I20260810 01:01:10.467612 4686 consensus_queue.cc:237] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825440956e2e4d4aa81efda8a3262648" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 35483 } } peers { permanent_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39567 } } peers { permanent_uuid: "85cb24da8f7941869b014c3b90e3b3da" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 46663 } }
I20260810 01:01:10.472638 4145 catalog_manager.cc:5495] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e reported cstate change: term changed from 0 to 1, leader changed from <none> to f4fff50a490a4e29bfa942a2bf11ff0e (127.20.81.131). New cstate: current_term: 1 leader_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825440956e2e4d4aa81efda8a3262648" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 35483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85cb24da8f7941869b014c3b90e3b3da" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 46663 } health_report { overall_health: UNKNOWN } } }
I20260810 01:01:10.489094 4679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:10.483641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50560 (local address 127.20.81.131:39567)
0810 01:01:10.483862 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:10.483868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:10.483888 (+ 20us) server_negotiation.cc:407] Connection header received
0810 01:01:10.484020 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:10.484024 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:10.484109 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:10.484267 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:01:10.484650 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.485405 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.486935 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.487332 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.487486 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:10.487999 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:01:10.488480 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:01:10.488646 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:10.488752 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 01:01:10.488829 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260810 01:01:10.493894 4511 tablet.cc:2369] T 32962e08eeb24f678200a1e2b53b3bfa P 825440956e2e4d4aa81efda8a3262648: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:01:10.497177 4309 raft_consensus.cc:1261] T 32962e08eeb24f678200a1e2b53b3bfa P 85cb24da8f7941869b014c3b90e3b3da [term 1 FOLLOWER]: Refusing update from remote peer f4fff50a490a4e29bfa942a2bf11ff0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:01:10.497231 4458 raft_consensus.cc:1261] T 32962e08eeb24f678200a1e2b53b3bfa P 825440956e2e4d4aa81efda8a3262648 [term 1 FOLLOWER]: Refusing update from remote peer f4fff50a490a4e29bfa942a2bf11ff0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:01:10.498176 4690 consensus_queue.cc:1035] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "825440956e2e4d4aa81efda8a3262648" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 35483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:01:10.498560 4686 consensus_queue.cc:1035] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "85cb24da8f7941869b014c3b90e3b3da" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 46663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:01:10.510180 4694 mvcc.cc:205] Tried to move back new op lower bound from 7316781754346893312 to 7316781754239365120. Current Snapshot: MvccSnapshot[applied={T|T < 7316781754346893312}]
I20260810 01:01:10.525664 4145 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:01:10.525890 4145 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:01:10.529461 4145 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268: Sending DeleteTablet for 3 replicas of tablet 32962e08eeb24f678200a1e2b53b3bfa
I20260810 01:01:10.530709 4434 tablet_service.cc:1514] Processing DeleteTablet for tablet 32962e08eeb24f678200a1e2b53b3bfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:01:10 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:58218
I20260810 01:01:10.530741 4589 tablet_service.cc:1514] Processing DeleteTablet for tablet 32962e08eeb24f678200a1e2b53b3bfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:01:10 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:50556
I20260810 01:01:10.530829 4287 tablet_service.cc:1514] Processing DeleteTablet for tablet 32962e08eeb24f678200a1e2b53b3bfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:01:10 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:42914
I20260810 01:01:10.533010 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 4214
I20260810 01:01:10.542289 4704 tablet_replica.cc:330] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e: stopping tablet replica
I20260810 01:01:10.542906 4704 raft_consensus.cc:2229] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e [term 1 LEADER]: Raft consensus shutting down.
I20260810 01:01:10.543722 4704 raft_consensus.cc:2258] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:01:10.544317 4705 tablet_replica.cc:330] T 32962e08eeb24f678200a1e2b53b3bfa P 825440956e2e4d4aa81efda8a3262648: stopping tablet replica
I20260810 01:01:10.545223 4705 raft_consensus.cc:2229] T 32962e08eeb24f678200a1e2b53b3bfa P 825440956e2e4d4aa81efda8a3262648 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:01:10.547173 4704 ts_tablet_manager.cc:1905] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:01:10.547711 4705 raft_consensus.cc:2258] T 32962e08eeb24f678200a1e2b53b3bfa P 825440956e2e4d4aa81efda8a3262648 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:01:10.548545 4706 tablet_replica.cc:330] T 32962e08eeb24f678200a1e2b53b3bfa P 85cb24da8f7941869b014c3b90e3b3da: stopping tablet replica
I20260810 01:01:10.548996 4706 raft_consensus.cc:2229] T 32962e08eeb24f678200a1e2b53b3bfa P 85cb24da8f7941869b014c3b90e3b3da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:01:10.549513 4706 raft_consensus.cc:2258] T 32962e08eeb24f678200a1e2b53b3bfa P 85cb24da8f7941869b014c3b90e3b3da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:01:10.551718 4705 ts_tablet_manager.cc:1905] T 32962e08eeb24f678200a1e2b53b3bfa P 825440956e2e4d4aa81efda8a3262648: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:01:10.551834 4706 ts_tablet_manager.cc:1905] T 32962e08eeb24f678200a1e2b53b3bfa P 85cb24da8f7941869b014c3b90e3b3da: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 01:01:10 dist-test-slave-gp6n krb5kdc[4095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.129@KRBTEST.COM
I20260810 01:01:10.559166 4706 ts_tablet_manager.cc:1918] T 32962e08eeb24f678200a1e2b53b3bfa P 85cb24da8f7941869b014c3b90e3b3da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:01:10.559348 4706 log.cc:1196] T 32962e08eeb24f678200a1e2b53b3bfa P 85cb24da8f7941869b014c3b90e3b3da: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/wal/wals/32962e08eeb24f678200a1e2b53b3bfa
I20260810 01:01:10.559940 4706 ts_tablet_manager.cc:1939] T 32962e08eeb24f678200a1e2b53b3bfa P 85cb24da8f7941869b014c3b90e3b3da: Deleting consensus metadata
I20260810 01:01:10.561964 4128 catalog_manager.cc:4841] TS 85cb24da8f7941869b014c3b90e3b3da (127.20.81.129:46663): tablet 32962e08eeb24f678200a1e2b53b3bfa (table test-table [id=caad1d3a7b9f443a90793b2bb9d557d7]) successfully deleted
I20260810 01:01:10.563103 4705 ts_tablet_manager.cc:1918] T 32962e08eeb24f678200a1e2b53b3bfa P 825440956e2e4d4aa81efda8a3262648: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:01:10.563292 4705 log.cc:1196] T 32962e08eeb24f678200a1e2b53b3bfa P 825440956e2e4d4aa81efda8a3262648: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/32962e08eeb24f678200a1e2b53b3bfa
I20260810 01:01:10.563748 4704 ts_tablet_manager.cc:1918] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:01:10.563866 4705 ts_tablet_manager.cc:1939] T 32962e08eeb24f678200a1e2b53b3bfa P 825440956e2e4d4aa81efda8a3262648: Deleting consensus metadata
I20260810 01:01:10.564038 4704 log.cc:1196] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/32962e08eeb24f678200a1e2b53b3bfa
I20260810 01:01:10.564658 4704 ts_tablet_manager.cc:1939] T 32962e08eeb24f678200a1e2b53b3bfa P f4fff50a490a4e29bfa942a2bf11ff0e: Deleting consensus metadata
I20260810 01:01:10.565721 4129 catalog_manager.cc:4841] TS 825440956e2e4d4aa81efda8a3262648 (127.20.81.130:35483): tablet 32962e08eeb24f678200a1e2b53b3bfa (table test-table [id=caad1d3a7b9f443a90793b2bb9d557d7]) successfully deleted
I20260810 01:01:10.566709 4129 catalog_manager.cc:4841] TS f4fff50a490a4e29bfa942a2bf11ff0e (127.20.81.131:39567): tablet 32962e08eeb24f678200a1e2b53b3bfa (table test-table [id=caad1d3a7b9f443a90793b2bb9d557d7]) successfully deleted
I20260810 01:01:10.570825 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:10.533806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42928 (local address 127.20.81.129:46663)
0810 01:01:10.533967 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:10.533973 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:10.541054 (+ 7081us) server_negotiation.cc:407] Connection header received
0810 01:01:10.541415 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:10.541424 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:10.541536 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:10.541628 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:10.548408 (+ 6780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.549658 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.552229 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.552903 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.553014 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:10.563683 (+ 10669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:10.563697 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:10.563717 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:10.563765 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:10.567757 (+ 3992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:10.568502 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:10.568512 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:10.568516 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:10.568684 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:10.569076 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:10.569081 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:10.569085 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:10.569561 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:10.569903 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:10.570144 (+ 241us) server_negotiation.cc:299] Negotiation successful
0810 01:01:10.570389 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 01:01:10.571477 4354 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:10.682512 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 4214
I20260810 01:01:10.705646 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 4365
Aug 10 01:01:10 dist-test-slave-gp6n krb5kdc[4095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.130@KRBTEST.COM
I20260810 01:01:10.723757 4675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:10.706549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58220 (local address 127.20.81.130:35483)
0810 01:01:10.706729 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:10.706734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:10.706759 (+ 25us) server_negotiation.cc:407] Connection header received
0810 01:01:10.707034 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:10.707039 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:10.707109 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:10.707201 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:10.708887 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.709739 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.711831 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.712233 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.712338 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:10.717774 (+ 5436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:10.717790 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:10.717806 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:10.717849 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:10.720816 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:10.721658 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:10.721664 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:10.721667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:10.721791 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:10.722189 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:10.722192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:10.722194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:10.722621 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:10.722918 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:10.723252 (+ 334us) server_negotiation.cc:299] Negotiation successful
0810 01:01:10.723419 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 01:01:10.724436 4504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:10.826572 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 4365
I20260810 01:01:10.849386 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 4516
Aug 10 01:01:10 dist-test-slave-gp6n krb5kdc[4095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.131@KRBTEST.COM
I20260810 01:01:10.868903 4679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:10.850226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50574 (local address 127.20.81.131:39567)
0810 01:01:10.850543 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:10.850551 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:10.850709 (+ 158us) server_negotiation.cc:407] Connection header received
0810 01:01:10.850954 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:10.850959 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:10.851044 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:10.851136 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:10.853097 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.853903 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.855816 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:10.856274 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:10.856371 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:10.862320 (+ 5949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:10.862335 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:10.862351 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:10.862394 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:10.865620 (+ 3226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:10.866556 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:10.866565 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:10.866569 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:10.866745 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:10.867196 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:10.867200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:10.867201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:10.867627 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:10.867869 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:10.868282 (+ 413us) server_negotiation.cc:299] Negotiation successful
0810 01:01:10.868506 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 01:01:10.869679 4654 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:10.997318 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 4516
I20260810 01:01:11.022058 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 4111
I20260810 01:01:11.023340 4180 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:11.136826 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 4111
I20260810 01:01:11.162751 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:38513
--webserver_interface=127.20.81.190
--webserver_port=43005
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:38513
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:01:11.509620 4714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:11.509912 4714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:11.510002 4714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:11.517130 4714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:01:11.517251 4714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:11.517298 4714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:11.517349 4714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:01:11.517395 4714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:01:11.527323 4714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:38513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:38513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=43005
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:11.528578 4714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:11.529881 4714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:01:11.557006 4725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:11.556918 4722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:11.556978 4721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:11.557924 4714 server_base.cc:1034] running on GCE node
I20260810 01:01:11.558892 4714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:11.560487 4714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:11.561743 4714 hybrid_clock.cc:648] HybridClock initialized: now 1786323671561694 us; error 52 us; skew 500 ppm
I20260810 01:01:11.562178 4714 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:11 dist-test-slave-gp6n krb5kdc[4095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323671, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:11.571081 4714 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 01:01:11.572829 4714 webserver.cc:460] Webserver started at http://127.20.81.190:43005/ using document root <none> and password file <none>
I20260810 01:01:11.573576 4714 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:11.573704 4714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:11.579733 4714 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 01:01:11.582588 4732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:11.583954 4714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:01:11.584131 4714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "5d1ff61743224f77bab36de848a90268"
format_stamp: "Formatted at 2026-08-10 01:01:08 on dist-test-slave-gp6n"
server_key: "a35b53fdf650ba0968fa4e9706fd7ec5"
server_key_iv: "23beb2b95b05319a3be4b96847b4e11a"
server_key_version: "encryptionkey@0"
I20260810 01:01:11.585198 4714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:11.605827 4714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:11.610479 4714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:11.610798 4714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:11.630028 4714 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:38513
I20260810 01:01:11.630048 4795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:38513 every 8 connection(s)
I20260810 01:01:11.631590 4714 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:01:11.638783 4796 sys_catalog.cc:263] Verifying existing consensus state
I20260810 01:01:11.642023 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 4714
I20260810 01:01:11.642500 4796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268: Bootstrap starting.
I20260810 01:01:11.643142 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:46663
--local_ip_for_outbound_sockets=127.20.81.129
--tserver_master_addrs=127.20.81.190:38513
--webserver_port=46621
--webserver_interface=127.20.81.129
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 01:01:11.666697 4796 log.cc:824] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268: Log is configured to *not* fsync() on all Append() calls
I20260810 01:01:11.679334 4796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 01:01:11.679903 4796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268: Bootstrap complete.
I20260810 01:01:11.689483 4796 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1ff61743224f77bab36de848a90268" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38513 } }
I20260810 01:01:11.690403 4796 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d1ff61743224f77bab36de848a90268, State: Initialized, Role: FOLLOWER
I20260810 01:01:11.691036 4796 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1ff61743224f77bab36de848a90268" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38513 } }
I20260810 01:01:11.691293 4796 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:01:11.691421 4796 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:01:11.691622 4796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 1 FOLLOWER]: Advancing to term 2
I20260810 01:01:11.695549 4796 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1ff61743224f77bab36de848a90268" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38513 } }
I20260810 01:01:11.696082 4796 leader_election.cc:302] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d1ff61743224f77bab36de848a90268; no voters:
I20260810 01:01:11.696725 4796 leader_election.cc:288] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 01:01:11.696894 4800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 01:01:11.698693 4800 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 2 LEADER]: Becoming Leader. State: Replica: 5d1ff61743224f77bab36de848a90268, State: Running, Role: LEADER
I20260810 01:01:11.699242 4796 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:01:11.699352 4800 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1ff61743224f77bab36de848a90268" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38513 } }
I20260810 01:01:11.702746 4802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d1ff61743224f77bab36de848a90268. Latest consensus state: current_term: 2 leader_uuid: "5d1ff61743224f77bab36de848a90268" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1ff61743224f77bab36de848a90268" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38513 } } }
I20260810 01:01:11.703212 4802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [sys.catalog]: This master's current role is: LEADER
I20260810 01:01:11.703792 4801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5d1ff61743224f77bab36de848a90268" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1ff61743224f77bab36de848a90268" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38513 } } }
I20260810 01:01:11.704130 4801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [sys.catalog]: This master's current role is: LEADER
I20260810 01:01:11.717306 4813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:01:11.725338 4813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:01:11.726833 4813 catalog_manager.cc:1259] Loaded cluster ID: 1fb0ad0d74ee445fbd32233456b2f9c2
I20260810 01:01:11.727006 4813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:01:11.730324 4813 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 01:01:11.748250 4813 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 01:01:11.757673 4813 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:01:11.759418 4813 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268: Loaded TSK: 0
I20260810 01:01:11.760942 4813 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 01:01:12.011687 4798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:12.011978 4798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:12.012127 4798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:12.019726 4798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:12.019837 4798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:12.020015 4798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 01:01:12.030004 4798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:46663
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=46621
--tserver_master_addrs=127.20.81.190:38513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:12.031386 4798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:12.032768 4798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:01:12.048046 4831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:12.048048 4828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:12.048048 4829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:12.048846 4798 server_base.cc:1034] running on GCE node
I20260810 01:01:12.049629 4798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:12.050920 4798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:12.052201 4798 hybrid_clock.cc:648] HybridClock initialized: now 1786323672052136 us; error 49 us; skew 500 ppm
I20260810 01:01:12.052606 4798 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:12 dist-test-slave-gp6n krb5kdc[4095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323672, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:12.056794 4798 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 01:01:12.058588 4798 webserver.cc:460] Webserver started at http://127.20.81.129:46621/ using document root <none> and password file <none>
I20260810 01:01:12.059309 4798 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:12.059461 4798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:12.065210 4798 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 01:01:12.068192 4841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:12.069727 4798 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 01:01:12.069926 4798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "85cb24da8f7941869b014c3b90e3b3da"
format_stamp: "Formatted at 2026-08-10 01:01:09 on dist-test-slave-gp6n"
server_key: "5660f6f65c146709066d31c050fa1925"
server_key_iv: "e0ec9e2db86e01d66dc94fed693fce33"
server_key_version: "encryptionkey@0"
I20260810 01:01:12.071008 4798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:12.100364 4798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:12.105160 4798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:12.105469 4798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:12.106596 4798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:01:12.107990 4798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:01:12.108116 4798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:12.108259 4798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:01:12.108359 4798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:12.143092 4798 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:46663
I20260810 01:01:12.143187 4968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:46663 every 8 connection(s)
I20260810 01:01:12.144574 4798 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 01:01:12.146207 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 4798
I20260810 01:01:12.147006 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:35483
--local_ip_for_outbound_sockets=127.20.81.130
--tserver_master_addrs=127.20.81.190:38513
--webserver_port=39021
--webserver_interface=127.20.81.130
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 01:01:12 dist-test-slave-gp6n krb5kdc[4095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323672, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:01:12.170908 4972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:12.147941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.129:59819)
0810 01:01:12.149057 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:12.149109 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:12.151841 (+ 2732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:12.153329 (+ 1488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:12.153343 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:12.154154 (+ 811us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:01:12.154987 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:12.155017 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:12.156687 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:12.156695 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:12.158778 (+ 2083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:12.158791 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:12.159871 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:12.159879 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:12.160164 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:12.160863 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:01:12.160885 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:01:12.163062 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:01:12.166476 (+ 3414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:12.166485 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:12.166500 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:12.166917 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:12.167381 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:12.167386 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:12.167390 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:12.167587 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:12.168362 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:01:12.168370 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:01:12.168701 (+ 331us) client_negotiation.cc:769] Sending connection context
0810 01:01:12.168958 (+ 257us) client_negotiation.cc:240] Negotiation successful
0810 01:01:12.169541 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"thread_start_us":605,"threads_started":1}
I20260810 01:01:12.170910 4971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:12.147921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:59819 (local address 127.20.81.190:38513)
0810 01:01:12.148702 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:12.148718 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:12.149264 (+ 546us) server_negotiation.cc:407] Connection header received
0810 01:01:12.152412 (+ 3148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:12.152460 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:12.152798 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:12.153190 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:12.155226 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:12.156507 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:12.158933 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:12.159609 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:12.159834 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:12.163340 (+ 3506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:12.163348 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:12.163362 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:12.163397 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:12.166297 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:12.167075 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:12.167082 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:12.167087 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:12.167250 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:12.167720 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:12.167725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:12.167727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:12.168189 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:12.168410 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:12.169192 (+ 782us) server_negotiation.cc:299] Negotiation successful
0810 01:01:12.169596 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":290,"threads_started":1}
I20260810 01:01:12.174387 4969 heartbeater.cc:347] Connected to a master server at 127.20.81.190:38513
I20260810 01:01:12.174762 4969 heartbeater.cc:464] Registering TS with master...
I20260810 01:01:12.175662 4969 heartbeater.cc:511] Master 127.20.81.190:38513 requested a full tablet report, sending...
I20260810 01:01:12.178224 4753 ts_manager.cc:194] Registered new tserver with Master: 85cb24da8f7941869b014c3b90e3b3da (127.20.81.129:46663)
I20260810 01:01:12.180315 4753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:59819
W20260810 01:01:12.505069 4974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:12.505427 4974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:12.505615 4974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:12.513106 4974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:12.513227 4974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:12.513345 4974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 01:01:12.523418 4974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:35483
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=39021
--tserver_master_addrs=127.20.81.190:38513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:12.524729 4974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:12.526127 4974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:01:12.539448 4984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:12.539732 4981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:12.539753 4982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:12.540686 4974 server_base.cc:1034] running on GCE node
I20260810 01:01:12.541280 4974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:12.542236 4974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:12.543470 4974 hybrid_clock.cc:648] HybridClock initialized: now 1786323672543415 us; error 71 us; skew 500 ppm
I20260810 01:01:12.543790 4974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:12 dist-test-slave-gp6n krb5kdc[4095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323672, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:12.547024 4974 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 01:01:12.548393 4974 webserver.cc:460] Webserver started at http://127.20.81.130:39021/ using document root <none> and password file <none>
I20260810 01:01:12.549045 4974 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:12.549162 4974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:12.554764 4974 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 01:01:12.557577 4995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:12.559058 4974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:01:12.559245 4974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "825440956e2e4d4aa81efda8a3262648"
format_stamp: "Formatted at 2026-08-10 01:01:09 on dist-test-slave-gp6n"
server_key: "ac6f1a60c97ca6f7ae4fb0a3be792c59"
server_key_iv: "c88e07166452f6dd82f9428cfbbe21b0"
server_key_version: "encryptionkey@0"
I20260810 01:01:12.560258 4974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:12.571215 4974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:12.575934 4974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:12.576228 4974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:12.577322 4974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:01:12.578771 4974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:01:12.578908 4974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:12.579056 4974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:01:12.579159 4974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:12.612609 4974 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:35483
I20260810 01:01:12.612703 5121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:35483 every 8 connection(s)
I20260810 01:01:12.614174 4974 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 01:01:12.617931 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 4974
I20260810 01:01:12.618693 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:39567
--local_ip_for_outbound_sockets=127.20.81.131
--tserver_master_addrs=127.20.81.190:38513
--webserver_port=34591
--webserver_interface=127.20.81.131
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 01:01:12 dist-test-slave-gp6n krb5kdc[4095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323672, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:01:12.636804 4971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:12.616928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:57045 (local address 127.20.81.190:38513)
0810 01:01:12.617119 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:12.617124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:12.618470 (+ 1346us) server_negotiation.cc:407] Connection header received
0810 01:01:12.620433 (+ 1963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:12.620442 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:12.620539 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:12.620663 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:12.622138 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:12.623303 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:12.624886 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:12.625631 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:12.625778 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:12.629369 (+ 3591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:12.629378 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:12.629386 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:12.629468 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:12.633217 (+ 3749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:12.633951 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:12.633957 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:12.633961 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:12.634147 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:12.634737 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:12.634743 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:12.634746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:12.635115 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:12.635257 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:12.636191 (+ 934us) server_negotiation.cc:299] Negotiation successful
0810 01:01:12.636440 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 01:01:12.638167 5124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:12.617416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.130:57045)
0810 01:01:12.618254 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:12.618302 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:12.620145 (+ 1843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:12.620868 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:12.620876 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:12.621383 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:01:12.621950 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:12.621967 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:12.623475 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:12.623478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:12.624717 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:12.624734 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:12.625841 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:12.625845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:12.626045 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:12.626798 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:01:12.626820 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:01:12.629075 (+ 2255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:01:12.633364 (+ 4289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:12.633371 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:12.633385 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:12.633824 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:12.634295 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:12.634302 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:12.634308 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:12.634602 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:12.635268 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:01:12.635279 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:01:12.635734 (+ 455us) client_negotiation.cc:769] Sending connection context
0810 01:01:12.636037 (+ 303us) client_negotiation.cc:240] Negotiation successful
0810 01:01:12.636659 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":351,"threads_started":1}
I20260810 01:01:12.639799 5122 heartbeater.cc:347] Connected to a master server at 127.20.81.190:38513
I20260810 01:01:12.640133 5122 heartbeater.cc:464] Registering TS with master...
I20260810 01:01:12.641016 5122 heartbeater.cc:511] Master 127.20.81.190:38513 requested a full tablet report, sending...
I20260810 01:01:12.642756 4753 ts_manager.cc:194] Registered new tserver with Master: 825440956e2e4d4aa81efda8a3262648 (127.20.81.130:35483)
I20260810 01:01:12.644047 4753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:57045
W20260810 01:01:12.978063 5125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:12.978333 5125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:12.978531 5125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:12.985625 5125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:12.985728 5125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:12.985948 5125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 01:01:12.995754 5125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:39567
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=34591
--tserver_master_addrs=127.20.81.190:38513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:12.997056 5125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:12.998451 5125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:01:13.011845 5130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:13.011909 5131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:13.011940 5133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:13.012843 5125 server_base.cc:1034] running on GCE node
I20260810 01:01:13.013406 5125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:13.014317 5125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:13.015530 5125 hybrid_clock.cc:648] HybridClock initialized: now 1786323673015463 us; error 40 us; skew 500 ppm
I20260810 01:01:13.015805 5125 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:13 dist-test-slave-gp6n krb5kdc[4095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323673, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:13.018877 5125 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 01:01:13.020221 5125 webserver.cc:460] Webserver started at http://127.20.81.131:34591/ using document root <none> and password file <none>
I20260810 01:01:13.020862 5125 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:13.020977 5125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:13.026144 5125 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 01:01:13.028976 5141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:13.030375 5125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 01:01:13.030586 5125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "f4fff50a490a4e29bfa942a2bf11ff0e"
format_stamp: "Formatted at 2026-08-10 01:01:10 on dist-test-slave-gp6n"
server_key: "79b4b035629a7d2ce6a4f116b22f93f9"
server_key_iv: "caba9d50c71fef6a89bd5d5015d1d5b6"
server_key_version: "encryptionkey@0"
I20260810 01:01:13.031544 5125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:13.049633 5125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:13.053846 5125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:13.054270 5125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:13.055476 5125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:01:13.056787 5125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:01:13.056874 5125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:13.057008 5125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:01:13.057106 5125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:13.091609 5125 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:39567
I20260810 01:01:13.091701 5272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:39567 every 8 connection(s)
I20260810 01:01:13.093159 5125 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 01:01:13.095579 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 5125
Aug 10 01:01:13 dist-test-slave-gp6n krb5kdc[4095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323673, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:01:13.114506 5275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:13.096545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49480 (local address 127.20.81.190:38513)
0810 01:01:13.096954 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:13.096959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:13.098993 (+ 2034us) server_negotiation.cc:407] Connection header received
0810 01:01:13.099151 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:13.099156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:13.099238 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:13.099392 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:13.101105 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.102125 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.103734 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.104379 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.104536 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:13.107491 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:13.107500 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:13.107508 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:13.107553 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:13.110905 (+ 3352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:13.112126 (+ 1221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:13.112130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:13.112132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:13.112285 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:13.112759 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:13.112762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:13.112765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:13.113201 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:13.113483 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:13.113756 (+ 273us) server_negotiation.cc:299] Negotiation successful
0810 01:01:13.113974 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":238,"threads_started":1}
I20260810 01:01:13.116396 4971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:13.096011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:48903 (local address 127.20.81.190:38513)
0810 01:01:13.096254 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:13.096259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:13.097636 (+ 1377us) server_negotiation.cc:407] Connection header received
0810 01:01:13.099806 (+ 2170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:13.099811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:13.099892 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:13.099994 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:13.101740 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.102725 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.104617 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.105299 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.105418 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:13.109247 (+ 3829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:13.109255 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:13.109262 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:13.109348 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:13.112756 (+ 3408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:13.113702 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:13.113707 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:13.113711 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:13.113872 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:13.114354 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:13.114358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:13.114360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:13.114776 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:13.114950 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:13.115834 (+ 884us) server_negotiation.cc:299] Negotiation successful
0810 01:01:13.116053 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"spinlock_wait_cycles":11008}
I20260810 01:01:13.117684 5277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:13.096659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.131:48903)
0810 01:01:13.097482 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:13.097523 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:13.099527 (+ 2004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:13.100250 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:13.100258 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:13.100802 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:01:13.101562 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:13.101588 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.102875 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.102879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:13.104462 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:13.104479 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.105526 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.105531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:13.105741 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:13.106395 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:01:13.106472 (+ 77us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:01:13.109001 (+ 2529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:01:13.112904 (+ 3903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:13.112915 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:13.112933 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:13.113556 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:13.114009 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:13.114013 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:13.114017 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:13.114252 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:13.114918 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:01:13.114927 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:01:13.115393 (+ 466us) client_negotiation.cc:769] Sending connection context
0810 01:01:13.115743 (+ 350us) client_negotiation.cc:240] Negotiation successful
0810 01:01:13.116325 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":362,"threads_started":1}
I20260810 01:01:13.119164 5273 heartbeater.cc:347] Connected to a master server at 127.20.81.190:38513
I20260810 01:01:13.119496 5273 heartbeater.cc:464] Registering TS with master...
I20260810 01:01:13.120244 5273 heartbeater.cc:511] Master 127.20.81.190:38513 requested a full tablet report, sending...
I20260810 01:01:13.121646 4753 ts_manager.cc:194] Registered new tserver with Master: f4fff50a490a4e29bfa942a2bf11ff0e (127.20.81.131:39567)
I20260810 01:01:13.122535 4753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:48903
I20260810 01:01:13.129542 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 01:01:13.136385 4971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:13.130466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49484 (local address 127.20.81.190:38513)
0810 01:01:13.130621 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:13.130626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:13.131131 (+ 505us) server_negotiation.cc:407] Connection header received
0810 01:01:13.131368 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:13.131371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:13.131527 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:13.131679 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:01:13.132121 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.132834 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.134274 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.134742 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.134875 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:13.135358 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:01:13.135817 (+ 459us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:01:13.135970 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:13.136068 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 01:01:13.136145 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 01:01:13.137229 4753 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:49484:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 01:01:13.144771 4753 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 01:01:13.169540 5283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:13.156031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:35483 (local address 127.0.0.1:48168)
0810 01:01:13.157180 (+ 1149us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:13.157226 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:13.157434 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:13.158700 (+ 1266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:13.158710 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:13.158753 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:01:13.159225 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:13.159242 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.161845 (+ 2603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.161853 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:13.165058 (+ 3205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:13.165081 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.168114 (+ 3033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.168124 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:13.168423 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:13.168507 (+ 84us) client_negotiation.cc:769] Sending connection context
0810 01:01:13.168680 (+ 173us) client_negotiation.cc:240] Negotiation successful
0810 01:01:13.168950 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"thread_start_us":1004,"threads_started":1}
I20260810 01:01:13.170960 5284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:13.156207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48168 (local address 127.20.81.130:35483)
0810 01:01:13.157002 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:13.157012 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:13.157681 (+ 669us) server_negotiation.cc:407] Connection header received
0810 01:01:13.157806 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:13.157816 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:13.158115 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:13.158337 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:01:13.159512 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.161617 (+ 2105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.165321 (+ 3704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.167410 (+ 2089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.167615 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:13.167908 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:13.170194 (+ 2286us) server_negotiation.cc:299] Negotiation successful
0810 01:01:13.170481 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":513,"threads_started":1}
I20260810 01:01:13.177448 5046 tablet_service.cc:1467] Processing CreateTablet for tablet ae3469d5e6bb4736baa734f6284ad1e1 (DEFAULT_TABLE table=test-table [id=b2ef08052427486dabba70a8a927276f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:01:13.178984 5046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae3469d5e6bb4736baa734f6284ad1e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:01:13.180224 5288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:13.158697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47364 (local address 127.20.81.131:39567)
0810 01:01:13.172347 (+ 13650us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:13.172354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:13.172380 (+ 26us) server_negotiation.cc:407] Connection header received
0810 01:01:13.172456 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:13.172461 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:13.172662 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:13.172870 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:01:13.173557 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.175095 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.177728 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.179135 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.179277 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:13.179560 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:13.179733 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 01:01:13.179890 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13495,"thread_start_us":13375,"threads_started":1}
I20260810 01:01:13.180274 5285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:13.157963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:39567 (local address 127.0.0.1:47364)
0810 01:01:13.171704 (+ 13741us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:13.171724 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:13.171840 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:13.172852 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:13.172857 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:13.172881 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:01:13.173354 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:13.173369 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.175283 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.175293 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:13.177522 (+ 2229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:13.177537 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.179339 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.179346 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:13.179541 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:13.179576 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 01:01:13.179678 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 01:01:13.179812 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13614,"mutex_wait_us":26,"thread_start_us":15310,"threads_started":1}
I20260810 01:01:13.180882 5283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:13.157017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:46663 (local address 127.0.0.1:45764)
0810 01:01:13.169829 (+ 12812us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:13.169849 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:13.169981 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:13.173763 (+ 3782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:13.173768 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:13.173798 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:01:13.174007 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:13.174020 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.176204 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.176209 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:13.178072 (+ 1863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:13.178098 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.179926 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.179933 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:13.180145 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:13.180175 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 01:01:13.180250 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 01:01:13.180405 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12720,"mutex_wait_us":3,"thread_start_us":14407,"threads_started":1}
I20260810 01:01:13.181340 5286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:13.157295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45764 (local address 127.20.81.129:46663)
0810 01:01:13.172796 (+ 15501us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:13.172808 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:13.172858 (+ 50us) server_negotiation.cc:407] Connection header received
0810 01:01:13.173015 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:13.173026 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:13.173314 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:13.174214 (+ 900us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:01:13.174489 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.176057 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.178262 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.179753 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.180555 (+ 802us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:13.180790 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:13.180861 (+ 71us) server_negotiation.cc:299] Negotiation successful
0810 01:01:13.181033 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15263,"thread_start_us":16177,"threads_started":1}
I20260810 01:01:13.183499 4969 heartbeater.cc:503] Master 127.20.81.190:38513 was elected leader, sending a full tablet report...
I20260810 01:01:13.184755 5194 tablet_service.cc:1467] Processing CreateTablet for tablet ae3469d5e6bb4736baa734f6284ad1e1 (DEFAULT_TABLE table=test-table [id=b2ef08052427486dabba70a8a927276f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:01:13.186247 5194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae3469d5e6bb4736baa734f6284ad1e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:01:13.188481 4898 tablet_service.cc:1467] Processing CreateTablet for tablet ae3469d5e6bb4736baa734f6284ad1e1 (DEFAULT_TABLE table=test-table [id=b2ef08052427486dabba70a8a927276f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:01:13.189854 4898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae3469d5e6bb4736baa734f6284ad1e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:01:13.194691 5289 tablet_bootstrap.cc:492] T ae3469d5e6bb4736baa734f6284ad1e1 P 825440956e2e4d4aa81efda8a3262648: Bootstrap starting.
I20260810 01:01:13.197994 5289 tablet_bootstrap.cc:654] T ae3469d5e6bb4736baa734f6284ad1e1 P 825440956e2e4d4aa81efda8a3262648: Neither blocks nor log segments found. Creating new log.
I20260810 01:01:13.199084 5289 log.cc:824] T ae3469d5e6bb4736baa734f6284ad1e1 P 825440956e2e4d4aa81efda8a3262648: Log is configured to *not* fsync() on all Append() calls
I20260810 01:01:13.201870 5291 tablet_bootstrap.cc:492] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e: Bootstrap starting.
I20260810 01:01:13.202064 5289 tablet_bootstrap.cc:492] T ae3469d5e6bb4736baa734f6284ad1e1 P 825440956e2e4d4aa81efda8a3262648: No bootstrap required, opened a new log
I20260810 01:01:13.202333 5289 ts_tablet_manager.cc:1397] T ae3469d5e6bb4736baa734f6284ad1e1 P 825440956e2e4d4aa81efda8a3262648: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260810 01:01:13.205385 5292 tablet_bootstrap.cc:492] T ae3469d5e6bb4736baa734f6284ad1e1 P 85cb24da8f7941869b014c3b90e3b3da: Bootstrap starting.
I20260810 01:01:13.205730 5291 tablet_bootstrap.cc:654] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e: Neither blocks nor log segments found. Creating new log.
I20260810 01:01:13.206990 5291 log.cc:824] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e: Log is configured to *not* fsync() on all Append() calls
I20260810 01:01:13.208449 5292 tablet_bootstrap.cc:654] T ae3469d5e6bb4736baa734f6284ad1e1 P 85cb24da8f7941869b014c3b90e3b3da: Neither blocks nor log segments found. Creating new log.
I20260810 01:01:13.209501 5292 log.cc:824] T ae3469d5e6bb4736baa734f6284ad1e1 P 85cb24da8f7941869b014c3b90e3b3da: Log is configured to *not* fsync() on all Append() calls
I20260810 01:01:13.210175 5291 tablet_bootstrap.cc:492] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e: No bootstrap required, opened a new log
I20260810 01:01:13.210500 5291 ts_tablet_manager.cc:1397] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260810 01:01:13.212146 5292 tablet_bootstrap.cc:492] T ae3469d5e6bb4736baa734f6284ad1e1 P 85cb24da8f7941869b014c3b90e3b3da: No bootstrap required, opened a new log
I20260810 01:01:13.212649 5292 ts_tablet_manager.cc:1397] T ae3469d5e6bb4736baa734f6284ad1e1 P 85cb24da8f7941869b014c3b90e3b3da: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260810 01:01:13.213332 5289 raft_consensus.cc:348] T ae3469d5e6bb4736baa734f6284ad1e1 P 825440956e2e4d4aa81efda8a3262648 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825440956e2e4d4aa81efda8a3262648" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 35483 } } peers { permanent_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39567 } } peers { permanent_uuid: "85cb24da8f7941869b014c3b90e3b3da" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 46663 } }
I20260810 01:01:13.213773 5289 raft_consensus.cc:374] T ae3469d5e6bb4736baa734f6284ad1e1 P 825440956e2e4d4aa81efda8a3262648 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:01:13.213871 5289 raft_consensus.cc:729] T ae3469d5e6bb4736baa734f6284ad1e1 P 825440956e2e4d4aa81efda8a3262648 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825440956e2e4d4aa81efda8a3262648, State: Initialized, Role: FOLLOWER
I20260810 01:01:13.214593 5289 consensus_queue.cc:260] T ae3469d5e6bb4736baa734f6284ad1e1 P 825440956e2e4d4aa81efda8a3262648 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825440956e2e4d4aa81efda8a3262648" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 35483 } } peers { permanent_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39567 } } peers { permanent_uuid: "85cb24da8f7941869b014c3b90e3b3da" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 46663 } }
I20260810 01:01:13.215751 5122 heartbeater.cc:503] Master 127.20.81.190:38513 was elected leader, sending a full tablet report...
I20260810 01:01:13.216405 5289 ts_tablet_manager.cc:1428] T ae3469d5e6bb4736baa734f6284ad1e1 P 825440956e2e4d4aa81efda8a3262648: Time spent starting tablet: real 0.014s user 0.007s sys 0.005s
I20260810 01:01:13.221037 5291 raft_consensus.cc:348] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825440956e2e4d4aa81efda8a3262648" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 35483 } } peers { permanent_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39567 } } peers { permanent_uuid: "85cb24da8f7941869b014c3b90e3b3da" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 46663 } }
I20260810 01:01:13.221398 5291 raft_consensus.cc:374] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:01:13.221520 5291 raft_consensus.cc:729] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4fff50a490a4e29bfa942a2bf11ff0e, State: Initialized, Role: FOLLOWER
I20260810 01:01:13.222232 5291 consensus_queue.cc:260] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825440956e2e4d4aa81efda8a3262648" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 35483 } } peers { permanent_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39567 } } peers { permanent_uuid: "85cb24da8f7941869b014c3b90e3b3da" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 46663 } }
I20260810 01:01:13.222234 5292 raft_consensus.cc:348] T ae3469d5e6bb4736baa734f6284ad1e1 P 85cb24da8f7941869b014c3b90e3b3da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825440956e2e4d4aa81efda8a3262648" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 35483 } } peers { permanent_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39567 } } peers { permanent_uuid: "85cb24da8f7941869b014c3b90e3b3da" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 46663 } }
I20260810 01:01:13.222641 5292 raft_consensus.cc:374] T ae3469d5e6bb4736baa734f6284ad1e1 P 85cb24da8f7941869b014c3b90e3b3da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:01:13.222764 5292 raft_consensus.cc:729] T ae3469d5e6bb4736baa734f6284ad1e1 P 85cb24da8f7941869b014c3b90e3b3da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85cb24da8f7941869b014c3b90e3b3da, State: Initialized, Role: FOLLOWER
I20260810 01:01:13.223520 5273 heartbeater.cc:503] Master 127.20.81.190:38513 was elected leader, sending a full tablet report...
I20260810 01:01:13.223353 5292 consensus_queue.cc:260] T ae3469d5e6bb4736baa734f6284ad1e1 P 85cb24da8f7941869b014c3b90e3b3da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825440956e2e4d4aa81efda8a3262648" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 35483 } } peers { permanent_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39567 } } peers { permanent_uuid: "85cb24da8f7941869b014c3b90e3b3da" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 46663 } }
I20260810 01:01:13.224261 5291 ts_tablet_manager.cc:1428] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e: Time spent starting tablet: real 0.014s user 0.008s sys 0.003s
I20260810 01:01:13.225392 5292 ts_tablet_manager.cc:1428] T ae3469d5e6bb4736baa734f6284ad1e1 P 85cb24da8f7941869b014c3b90e3b3da: Time spent starting tablet: real 0.013s user 0.007s sys 0.005s
W20260810 01:01:13.344870 5274 tablet.cc:2369] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 01:01:13.366250 5123 tablet.cc:2369] T ae3469d5e6bb4736baa734f6284ad1e1 P 825440956e2e4d4aa81efda8a3262648: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 01:01:13.397104 4970 tablet.cc:2369] T ae3469d5e6bb4736baa734f6284ad1e1 P 85cb24da8f7941869b014c3b90e3b3da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:01:13.544359 5298 raft_consensus.cc:482] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:01:13.544682 5298 raft_consensus.cc:504] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825440956e2e4d4aa81efda8a3262648" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 35483 } } peers { permanent_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39567 } } peers { permanent_uuid: "85cb24da8f7941869b014c3b90e3b3da" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 46663 } }
I20260810 01:01:13.546516 5298 leader_election.cc:288] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 825440956e2e4d4aa81efda8a3262648 (127.20.81.130:35483), 85cb24da8f7941869b014c3b90e3b3da (127.20.81.129:46663)
I20260810 01:01:13.552395 5284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:13.547244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:49347 (local address 127.20.81.130:35483)
0810 01:01:13.547440 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:13.547445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:13.547468 (+ 23us) server_negotiation.cc:407] Connection header received
0810 01:01:13.547530 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:13.547533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:13.547589 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:13.547697 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:01:13.548147 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.548847 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.550651 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.551485 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.551648 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:13.551725 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:13.551990 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 01:01:13.552122 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 01:01:13.552474 5277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:13.546767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:35483 (local address 127.20.81.131:49347)
0810 01:01:13.547067 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:13.547091 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:13.547226 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:13.547721 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:13.547726 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:13.547756 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:01:13.547990 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:13.547999 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.548988 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.548993 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:13.550497 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:13.550509 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.551638 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.551644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:13.551826 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:13.551849 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 01:01:13.551953 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 01:01:13.552127 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":40,"spinlock_wait_cycles":32640}
I20260810 01:01:13.552839 5286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:13.547239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:34061 (local address 127.20.81.129:46663)
0810 01:01:13.547440 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:13.547448 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:13.547781 (+ 333us) server_negotiation.cc:407] Connection header received
0810 01:01:13.548016 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:13.548020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:13.548078 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:13.548174 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:01:13.548687 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.549407 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.551159 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.551935 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.552128 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:13.552198 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:13.552461 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 01:01:13.552591 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 01:01:13.552963 5300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:13.547008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:46663 (local address 127.20.81.131:34061)
0810 01:01:13.547665 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:13.547690 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:13.547862 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:13.548188 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:13.548192 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:13.548214 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:01:13.548539 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:13.548548 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.549556 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.549563 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:13.551009 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:13.551021 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.552142 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.552147 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:13.552306 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:13.552330 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 01:01:13.552425 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 01:01:13.552568 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"mutex_wait_us":19,"thread_start_us":423,"threads_started":1}
I20260810 01:01:13.553690 5071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae3469d5e6bb4736baa734f6284ad1e1" candidate_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825440956e2e4d4aa81efda8a3262648" is_pre_election: true
I20260810 01:01:13.554132 4920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae3469d5e6bb4736baa734f6284ad1e1" candidate_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85cb24da8f7941869b014c3b90e3b3da" is_pre_election: true
I20260810 01:01:13.554307 5071 raft_consensus.cc:2454] T ae3469d5e6bb4736baa734f6284ad1e1 P 825440956e2e4d4aa81efda8a3262648 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4fff50a490a4e29bfa942a2bf11ff0e in term 0.
I20260810 01:01:13.554751 4920 raft_consensus.cc:2454] T ae3469d5e6bb4736baa734f6284ad1e1 P 85cb24da8f7941869b014c3b90e3b3da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4fff50a490a4e29bfa942a2bf11ff0e in term 0.
I20260810 01:01:13.555140 5144 leader_election.cc:302] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 825440956e2e4d4aa81efda8a3262648, f4fff50a490a4e29bfa942a2bf11ff0e; no voters:
I20260810 01:01:13.555572 5298 raft_consensus.cc:2781] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:01:13.555721 5298 raft_consensus.cc:482] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:01:13.555840 5298 raft_consensus.cc:3037] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:01:13.558075 5298 raft_consensus.cc:504] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825440956e2e4d4aa81efda8a3262648" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 35483 } } peers { permanent_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39567 } } peers { permanent_uuid: "85cb24da8f7941869b014c3b90e3b3da" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 46663 } }
I20260810 01:01:13.558781 5298 leader_election.cc:288] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e [CANDIDATE]: Term 1 election: Requested vote from peers 825440956e2e4d4aa81efda8a3262648 (127.20.81.130:35483), 85cb24da8f7941869b014c3b90e3b3da (127.20.81.129:46663)
I20260810 01:01:13.559154 5071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae3469d5e6bb4736baa734f6284ad1e1" candidate_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825440956e2e4d4aa81efda8a3262648"
I20260810 01:01:13.559216 4920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae3469d5e6bb4736baa734f6284ad1e1" candidate_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85cb24da8f7941869b014c3b90e3b3da"
I20260810 01:01:13.559356 5071 raft_consensus.cc:3037] T ae3469d5e6bb4736baa734f6284ad1e1 P 825440956e2e4d4aa81efda8a3262648 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:01:13.559446 4920 raft_consensus.cc:3037] T ae3469d5e6bb4736baa734f6284ad1e1 P 85cb24da8f7941869b014c3b90e3b3da [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:01:13.561609 5071 raft_consensus.cc:2454] T ae3469d5e6bb4736baa734f6284ad1e1 P 825440956e2e4d4aa81efda8a3262648 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4fff50a490a4e29bfa942a2bf11ff0e in term 1.
I20260810 01:01:13.561659 4920 raft_consensus.cc:2454] T ae3469d5e6bb4736baa734f6284ad1e1 P 85cb24da8f7941869b014c3b90e3b3da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4fff50a490a4e29bfa942a2bf11ff0e in term 1.
I20260810 01:01:13.562008 5144 leader_election.cc:302] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 825440956e2e4d4aa81efda8a3262648, f4fff50a490a4e29bfa942a2bf11ff0e; no voters:
I20260810 01:01:13.562288 5298 raft_consensus.cc:2781] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:01:13.562763 5298 raft_consensus.cc:686] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e [term 1 LEADER]: Becoming Leader. State: Replica: f4fff50a490a4e29bfa942a2bf11ff0e, State: Running, Role: LEADER
I20260810 01:01:13.563333 5298 consensus_queue.cc:237] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825440956e2e4d4aa81efda8a3262648" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 35483 } } peers { permanent_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39567 } } peers { permanent_uuid: "85cb24da8f7941869b014c3b90e3b3da" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 46663 } }
I20260810 01:01:13.567883 4753 catalog_manager.cc:5495] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e reported cstate change: term changed from 0 to 1, leader changed from <none> to f4fff50a490a4e29bfa942a2bf11ff0e (127.20.81.131). New cstate: current_term: 1 leader_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825440956e2e4d4aa81efda8a3262648" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 35483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4fff50a490a4e29bfa942a2bf11ff0e" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 39567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85cb24da8f7941869b014c3b90e3b3da" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 46663 } health_report { overall_health: UNKNOWN } } }
I20260810 01:01:13.640429 5288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:13.634159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47370 (local address 127.20.81.131:39567)
0810 01:01:13.634365 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:13.634371 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:13.634395 (+ 24us) server_negotiation.cc:407] Connection header received
0810 01:01:13.634696 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:13.634702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:13.634800 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:13.634958 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:01:13.635257 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.636438 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.637937 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.638586 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.638704 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:13.639242 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:01:13.639859 (+ 617us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:01:13.640034 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:13.640081 (+ 47us) server_negotiation.cc:299] Negotiation successful
0810 01:01:13.640161 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 01:01:13.649571 4920 raft_consensus.cc:1261] T ae3469d5e6bb4736baa734f6284ad1e1 P 85cb24da8f7941869b014c3b90e3b3da [term 1 FOLLOWER]: Refusing update from remote peer f4fff50a490a4e29bfa942a2bf11ff0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:01:13.649700 5071 raft_consensus.cc:1261] T ae3469d5e6bb4736baa734f6284ad1e1 P 825440956e2e4d4aa81efda8a3262648 [term 1 FOLLOWER]: Refusing update from remote peer f4fff50a490a4e29bfa942a2bf11ff0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:01:13.650705 5301 consensus_queue.cc:1035] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "85cb24da8f7941869b014c3b90e3b3da" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 46663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:01:13.651086 5298 consensus_queue.cc:1035] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "825440956e2e4d4aa81efda8a3262648" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 35483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:01:13.662032 5303 mvcc.cc:205] Tried to move back new op lower bound from 7316781767258284032 to 7316781766919540736. Current Snapshot: MvccSnapshot[applied={T|T < 7316781767258284032}]
I20260810 01:01:13.677371 4753 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:49484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:01:13.677654 4753 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:49484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:01:13.681273 4753 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268: Sending DeleteTablet for 3 replicas of tablet ae3469d5e6bb4736baa734f6284ad1e1
I20260810 01:01:13.682488 5194 tablet_service.cc:1514] Processing DeleteTablet for tablet ae3469d5e6bb4736baa734f6284ad1e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:01:13 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:47364
I20260810 01:01:13.682468 5046 tablet_service.cc:1514] Processing DeleteTablet for tablet ae3469d5e6bb4736baa734f6284ad1e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:01:13 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:48168
I20260810 01:01:13.682513 4898 tablet_service.cc:1514] Processing DeleteTablet for tablet ae3469d5e6bb4736baa734f6284ad1e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:01:13 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:45764
I20260810 01:01:13.683606 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 4798
I20260810 01:01:13.704286 5313 tablet_replica.cc:330] T ae3469d5e6bb4736baa734f6284ad1e1 P 825440956e2e4d4aa81efda8a3262648: stopping tablet replica
I20260810 01:01:13.704929 5313 raft_consensus.cc:2229] T ae3469d5e6bb4736baa734f6284ad1e1 P 825440956e2e4d4aa81efda8a3262648 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:01:13.705627 5313 raft_consensus.cc:2258] T ae3469d5e6bb4736baa734f6284ad1e1 P 825440956e2e4d4aa81efda8a3262648 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:01:13.705021 5314 tablet_replica.cc:330] T ae3469d5e6bb4736baa734f6284ad1e1 P 85cb24da8f7941869b014c3b90e3b3da: stopping tablet replica
I20260810 01:01:13.706233 5314 raft_consensus.cc:2229] T ae3469d5e6bb4736baa734f6284ad1e1 P 85cb24da8f7941869b014c3b90e3b3da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:01:13.708329 5313 ts_tablet_manager.cc:1905] T ae3469d5e6bb4736baa734f6284ad1e1 P 825440956e2e4d4aa81efda8a3262648: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:01:13.708995 5314 raft_consensus.cc:2258] T ae3469d5e6bb4736baa734f6284ad1e1 P 85cb24da8f7941869b014c3b90e3b3da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:01:13.709040 5312 tablet_replica.cc:330] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e: stopping tablet replica
I20260810 01:01:13.709475 5312 raft_consensus.cc:2229] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e [term 1 LEADER]: Raft consensus shutting down.
I20260810 01:01:13.710191 5312 raft_consensus.cc:2258] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:01:13.712266 5314 ts_tablet_manager.cc:1905] T ae3469d5e6bb4736baa734f6284ad1e1 P 85cb24da8f7941869b014c3b90e3b3da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:01:13.715343 5313 ts_tablet_manager.cc:1918] T ae3469d5e6bb4736baa734f6284ad1e1 P 825440956e2e4d4aa81efda8a3262648: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:01:13.715526 5313 log.cc:1196] T ae3469d5e6bb4736baa734f6284ad1e1 P 825440956e2e4d4aa81efda8a3262648: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/ae3469d5e6bb4736baa734f6284ad1e1
I20260810 01:01:13.716071 5313 ts_tablet_manager.cc:1939] T ae3469d5e6bb4736baa734f6284ad1e1 P 825440956e2e4d4aa81efda8a3262648: Deleting consensus metadata
I20260810 01:01:13.717583 5312 ts_tablet_manager.cc:1905] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:01:13.718030 4737 catalog_manager.cc:4841] TS 825440956e2e4d4aa81efda8a3262648 (127.20.81.130:35483): tablet ae3469d5e6bb4736baa734f6284ad1e1 (table test-table [id=b2ef08052427486dabba70a8a927276f]) successfully deleted
I20260810 01:01:13.719933 5314 ts_tablet_manager.cc:1918] T ae3469d5e6bb4736baa734f6284ad1e1 P 85cb24da8f7941869b014c3b90e3b3da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:01:13.720163 5314 log.cc:1196] T ae3469d5e6bb4736baa734f6284ad1e1 P 85cb24da8f7941869b014c3b90e3b3da: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/wal/wals/ae3469d5e6bb4736baa734f6284ad1e1
I20260810 01:01:13.720707 5314 ts_tablet_manager.cc:1939] T ae3469d5e6bb4736baa734f6284ad1e1 P 85cb24da8f7941869b014c3b90e3b3da: Deleting consensus metadata
I20260810 01:01:13.722474 4736 catalog_manager.cc:4841] TS 85cb24da8f7941869b014c3b90e3b3da (127.20.81.129:46663): tablet ae3469d5e6bb4736baa734f6284ad1e1 (table test-table [id=b2ef08052427486dabba70a8a927276f]) successfully deleted
I20260810 01:01:13.723779 5286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:13.685097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45780 (local address 127.20.81.129:46663)
0810 01:01:13.685332 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:13.685342 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:13.694630 (+ 9288us) server_negotiation.cc:407] Connection header received
0810 01:01:13.695182 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:13.695191 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:13.695312 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:13.695403 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:13.698216 (+ 2813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.699830 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.702609 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.703565 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.703694 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:13.710575 (+ 6881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:13.710589 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:13.710606 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:13.710660 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:13.715253 (+ 4593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:13.718620 (+ 3367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:13.718631 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:13.718636 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:13.718821 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:13.719285 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:13.719290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:13.719294 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:13.719771 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:13.720127 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:13.723137 (+ 3010us) server_negotiation.cc:299] Negotiation successful
0810 01:01:13.723391 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 01:01:13.724637 4962 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:13.725440 5312 ts_tablet_manager.cc:1918] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:01:13.725651 5312 log.cc:1196] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/ae3469d5e6bb4736baa734f6284ad1e1
I20260810 01:01:13.726230 5312 ts_tablet_manager.cc:1939] T ae3469d5e6bb4736baa734f6284ad1e1 P f4fff50a490a4e29bfa942a2bf11ff0e: Deleting consensus metadata
I20260810 01:01:13.728528 4737 catalog_manager.cc:4841] TS f4fff50a490a4e29bfa942a2bf11ff0e (127.20.81.131:39567): tablet ae3469d5e6bb4736baa734f6284ad1e1 (table test-table [id=b2ef08052427486dabba70a8a927276f]) successfully deleted
I20260810 01:01:13.814580 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 4798
I20260810 01:01:13.844003 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 4974
I20260810 01:01:13.858486 5284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:13.844862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48178 (local address 127.20.81.130:35483)
0810 01:01:13.845064 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:13.845069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:13.845091 (+ 22us) server_negotiation.cc:407] Connection header received
0810 01:01:13.845301 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:13.845306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:13.845378 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:13.845489 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:13.846950 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.847686 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.849034 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.849448 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.849583 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:13.852400 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:13.852415 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:13.852433 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:13.852471 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:13.855382 (+ 2911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:13.856139 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:13.856147 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:13.856149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:13.856286 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:13.856685 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:13.856691 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:13.856694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:13.857181 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:13.857484 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:13.857837 (+ 353us) server_negotiation.cc:299] Negotiation successful
0810 01:01:13.858083 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 01:01:13.859124 5117 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:13.951915 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 4974
I20260810 01:01:13.975003 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 5125
I20260810 01:01:13.991971 5288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:13.975923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47374 (local address 127.20.81.131:39567)
0810 01:01:13.976193 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:13.976198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:13.976331 (+ 133us) server_negotiation.cc:407] Connection header received
0810 01:01:13.976537 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:13.976541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:13.976602 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:13.976702 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:13.978604 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.979447 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.981431 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:13.981896 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:13.982002 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:13.985691 (+ 3689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:13.985702 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:13.985716 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:13.985747 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:13.988755 (+ 3008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:13.989691 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:13.989698 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:13.989700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:13.989840 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:13.990287 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:13.990290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:13.990292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:13.990828 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:13.991071 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:13.991449 (+ 378us) server_negotiation.cc:299] Negotiation successful
0810 01:01:13.991613 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 01:01:13.992672 5268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:14.117554 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 5125
I20260810 01:01:14.142617 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 4714
I20260810 01:01:14.143985 4790 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:14.264146 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 4714
I20260810 01:01:14.291152 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:38513
--webserver_interface=127.20.81.190
--webserver_port=43005
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:38513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:01:14.636792 5321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:14.637173 5321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:14.637342 5321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:14.644716 5321 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:01:14.644886 5321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:14.645009 5321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:14.645108 5321 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:01:14.645193 5321 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:01:14.655480 5321 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:38513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:38513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=43005
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:14.656937 5321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:14.658505 5321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:01:14.669734 5326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:14.670511 5327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:14.671241 5329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:14.672016 5321 server_base.cc:1034] running on GCE node
I20260810 01:01:14.672828 5321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:14.674443 5321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:14.675719 5321 hybrid_clock.cc:648] HybridClock initialized: now 1786323674675656 us; error 58 us; skew 500 ppm
I20260810 01:01:14.676103 5321 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:14 dist-test-slave-gp6n krb5kdc[4095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323674, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:14.679418 5321 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 01:01:14.680833 5321 webserver.cc:460] Webserver started at http://127.20.81.190:43005/ using document root <none> and password file <none>
I20260810 01:01:14.681497 5321 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:14.681600 5321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:14.687000 5321 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 01:01:14.689743 5336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:14.691141 5321 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:01:14.691277 5321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "5d1ff61743224f77bab36de848a90268"
format_stamp: "Formatted at 2026-08-10 01:01:08 on dist-test-slave-gp6n"
server_key: "a35b53fdf650ba0968fa4e9706fd7ec5"
server_key_iv: "23beb2b95b05319a3be4b96847b4e11a"
server_key_version: "encryptionkey@0"
I20260810 01:01:14.692193 5321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:14.716017 5321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:14.720227 5321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:14.720463 5321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:14.739741 5321 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:38513
I20260810 01:01:14.739815 5394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:38513 every 8 connection(s)
I20260810 01:01:14.741303 5321 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:01:14.748333 5395 sys_catalog.cc:263] Verifying existing consensus state
I20260810 01:01:14.750380 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 5321
I20260810 01:01:14.751431 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:46663
--local_ip_for_outbound_sockets=127.20.81.129
--tserver_master_addrs=127.20.81.190:38513
--webserver_port=46621
--webserver_interface=127.20.81.129
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 01:01:14.752468 5395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268: Bootstrap starting.
I20260810 01:01:14.776660 5395 log.cc:824] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268: Log is configured to *not* fsync() on all Append() calls
I20260810 01:01:14.795670 5395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 01:01:14.796499 5395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268: Bootstrap complete.
I20260810 01:01:14.812709 5395 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1ff61743224f77bab36de848a90268" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38513 } }
I20260810 01:01:14.814018 5395 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d1ff61743224f77bab36de848a90268, State: Initialized, Role: FOLLOWER
I20260810 01:01:14.814849 5395 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1ff61743224f77bab36de848a90268" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38513 } }
I20260810 01:01:14.815187 5395 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:01:14.815337 5395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:01:14.815582 5395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 2 FOLLOWER]: Advancing to term 3
I20260810 01:01:14.823336 5395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1ff61743224f77bab36de848a90268" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38513 } }
I20260810 01:01:14.823992 5395 leader_election.cc:302] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d1ff61743224f77bab36de848a90268; no voters:
I20260810 01:01:14.824774 5395 leader_election.cc:288] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 01:01:14.824916 5399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 01:01:14.826719 5399 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [term 3 LEADER]: Becoming Leader. State: Replica: 5d1ff61743224f77bab36de848a90268, State: Running, Role: LEADER
I20260810 01:01:14.827283 5395 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:01:14.827354 5399 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1ff61743224f77bab36de848a90268" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38513 } }
I20260810 01:01:14.830691 5400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "5d1ff61743224f77bab36de848a90268" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1ff61743224f77bab36de848a90268" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38513 } } }
I20260810 01:01:14.831036 5400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [sys.catalog]: This master's current role is: LEADER
I20260810 01:01:14.830044 5401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d1ff61743224f77bab36de848a90268. Latest consensus state: current_term: 3 leader_uuid: "5d1ff61743224f77bab36de848a90268" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1ff61743224f77bab36de848a90268" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 38513 } } }
I20260810 01:01:14.831529 5401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d1ff61743224f77bab36de848a90268 [sys.catalog]: This master's current role is: LEADER
I20260810 01:01:14.851755 5406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:01:14.860752 5406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:01:14.866307 5406 catalog_manager.cc:1259] Loaded cluster ID: 1fb0ad0d74ee445fbd32233456b2f9c2
I20260810 01:01:14.866565 5406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 01:01:14.868954 5406 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f31130549ad google::LogMessage::Fail() at ??:0
@ 0x7f311305788a google::LogMessage::SendToLog() at ??:0
@ 0x7f31130544de google::LogMessage::Flush() at ??:0
@ 0x7f3113055a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3128a8d512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f3128a8a976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3128b12610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f3128b12439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3115ca0345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3115ca391c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f3115ca3799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3115c7eeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3118f586db start_thread at ??:0
@ 0x7f3111eee61f clone at ??:0
W20260810 01:01:15.204324 5397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:15.204659 5397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:15.204833 5397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:15.212353 5397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:15.212533 5397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:15.212716 5397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 01:01:15.222769 5397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:46663
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=46621
--tserver_master_addrs=127.20.81.190:38513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:15.224493 5397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:15.225981 5397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:01:15.240381 5439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:15.240479 5440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:15.240908 5442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:15.241469 5397 server_base.cc:1034] running on GCE node
I20260810 01:01:15.242105 5397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:15.243144 5397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:15.244395 5397 hybrid_clock.cc:648] HybridClock initialized: now 1786323675244343 us; error 48 us; skew 500 ppm
I20260810 01:01:15.244746 5397 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:15 dist-test-slave-gp6n krb5kdc[4095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323675, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:15.248000 5397 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 01:01:15.249411 5397 webserver.cc:460] Webserver started at http://127.20.81.129:46621/ using document root <none> and password file <none>
I20260810 01:01:15.250069 5397 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:15.250174 5397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:15.255543 5397 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 01:01:15.258154 5450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:15.259526 5397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 01:01:15.259702 5397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "85cb24da8f7941869b014c3b90e3b3da"
format_stamp: "Formatted at 2026-08-10 01:01:09 on dist-test-slave-gp6n"
server_key: "5660f6f65c146709066d31c050fa1925"
server_key_iv: "e0ec9e2db86e01d66dc94fed693fce33"
server_key_version: "encryptionkey@0"
I20260810 01:01:15.260681 5397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:15.282047 5397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:15.286208 5397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:15.286535 5397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:15.287878 5397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:01:15.289381 5397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:01:15.289486 5397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:15.289646 5397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:01:15.289744 5397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:15.326711 5397 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:46663
I20260810 01:01:15.326769 5580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:46663 every 8 connection(s)
I20260810 01:01:15.328256 5397 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 01:01:15.333163 5583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:15.331315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.129:52555)
0810 01:01:15.332066 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:15.332293 (+ 227us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":290,"threads_started":1}
W20260810 01:01:15.333909 5581 heartbeater.cc:644] Failed to heartbeat to 127.20.81.190:38513 (0 consecutive failures): Network error: Failed to ping master at 127.20.81.190:38513: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
I20260810 01:01:15.334785 5583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:15.334448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.129:58363)
0810 01:01:15.334608 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:15.334684 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 01:01:15.335714 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 5397
I20260810 01:01:15.335745 5583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:15.335419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.129:59465)
0810 01:01:15.335556 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:15.335633 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 01:01:15.336055 5581 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 01:01:15.336498 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:35483
--local_ip_for_outbound_sockets=127.20.81.130
--tserver_master_addrs=127.20.81.190:38513
--webserver_port=39021
--webserver_interface=127.20.81.130
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:01:15.679394 5584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:15.679666 5584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:15.679872 5584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:15.687227 5584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:15.687330 5584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:15.687523 5584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 01:01:15.697335 5584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:35483
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=39021
--tserver_master_addrs=127.20.81.190:38513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:15.698694 5584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:15.700100 5584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:01:15.713780 5592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:15.713855 5590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:15.713925 5589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:15.714747 5584 server_base.cc:1034] running on GCE node
I20260810 01:01:15.715400 5584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:15.716355 5584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:15.717593 5584 hybrid_clock.cc:648] HybridClock initialized: now 1786323675717526 us; error 58 us; skew 500 ppm
I20260810 01:01:15.717972 5584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:15 dist-test-slave-gp6n krb5kdc[4095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323675, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:15.721355 5584 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 01:01:15.723001 5584 webserver.cc:460] Webserver started at http://127.20.81.130:39021/ using document root <none> and password file <none>
I20260810 01:01:15.723770 5584 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:15.723917 5584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:15.729424 5584 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 01:01:15.732153 5601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:15.733479 5584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 01:01:15.733657 5584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "825440956e2e4d4aa81efda8a3262648"
format_stamp: "Formatted at 2026-08-10 01:01:09 on dist-test-slave-gp6n"
server_key: "ac6f1a60c97ca6f7ae4fb0a3be792c59"
server_key_iv: "c88e07166452f6dd82f9428cfbbe21b0"
server_key_version: "encryptionkey@0"
I20260810 01:01:15.734787 5584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:15.757577 5584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:15.762079 5584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:15.762364 5584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:15.763490 5584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:01:15.765035 5584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:01:15.765156 5584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:15.765292 5584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:01:15.765380 5584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:15.799484 5584 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:35483
I20260810 01:01:15.799577 5728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:35483 every 8 connection(s)
I20260810 01:01:15.801059 5584 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 01:01:15.805500 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 5584
I20260810 01:01:15.805984 5731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:15.804049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.130:55205)
0810 01:01:15.804765 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:15.805005 (+ 240us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":266,"threads_started":1}
I20260810 01:01:15.806217 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:39567
--local_ip_for_outbound_sockets=127.20.81.131
--tserver_master_addrs=127.20.81.190:38513
--webserver_port=34591
--webserver_interface=127.20.81.131
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:01:15.806684 5729 heartbeater.cc:644] Failed to heartbeat to 127.20.81.190:38513 (0 consecutive failures): Network error: Failed to ping master at 127.20.81.190:38513: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
I20260810 01:01:15.807543 5731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:15.807195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.130:50459)
0810 01:01:15.807355 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:15.807437 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260810 01:01:15.808456 5731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:15.808126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.130:56575)
0810 01:01:15.808271 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:15.808351 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 01:01:15.808722 5729 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 01:01:16.149168 5732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:16.149466 5732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:16.149626 5732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:16.156917 5732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:16.157038 5732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:16.157254 5732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 01:01:16.167402 5732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:39567
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=34591
--tserver_master_addrs=127.20.81.190:38513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:16.168885 5732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:16.170562 5732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:01:16.184024 5738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:16.183970 5739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:16.183928 5741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:16.184412 5732 server_base.cc:1034] running on GCE node
I20260810 01:01:16.185256 5732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:16.186192 5732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:16.187414 5732 hybrid_clock.cc:648] HybridClock initialized: now 1786323676187362 us; error 45 us; skew 500 ppm
I20260810 01:01:16.187736 5732 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:16 dist-test-slave-gp6n krb5kdc[4095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323676, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:16.190901 5732 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 01:01:16.192376 5732 webserver.cc:460] Webserver started at http://127.20.81.131:34591/ using document root <none> and password file <none>
I20260810 01:01:16.193065 5732 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:16.193223 5732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:16.198704 5732 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260810 01:01:16.201303 5748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:16.202610 5732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:01:16.202771 5732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "f4fff50a490a4e29bfa942a2bf11ff0e"
format_stamp: "Formatted at 2026-08-10 01:01:10 on dist-test-slave-gp6n"
server_key: "79b4b035629a7d2ce6a4f116b22f93f9"
server_key_iv: "caba9d50c71fef6a89bd5d5015d1d5b6"
server_key_version: "encryptionkey@0"
I20260810 01:01:16.203728 5732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:16.224270 5732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:16.228541 5732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:16.228760 5732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:16.229655 5732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:01:16.230944 5732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:01:16.231040 5732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:16.231123 5732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:01:16.231185 5732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:16.271703 5732 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:39567
I20260810 01:01:16.271822 5875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:39567 every 8 connection(s)
I20260810 01:01:16.273254 5732 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 01:01:16.275458 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 5732
I20260810 01:01:16.278529 5878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:16.276277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.131:41025)
0810 01:01:16.277116 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:16.277363 (+ 247us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":363,"threads_started":1}
W20260810 01:01:16.279284 5876 heartbeater.cc:644] Failed to heartbeat to 127.20.81.190:38513 (0 consecutive failures): Network error: Failed to ping master at 127.20.81.190:38513: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
I20260810 01:01:16.280030 5878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:16.279717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.131:37389)
0810 01:01:16.279869 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:16.279932 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 01:01:16.280946 5878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:16.280573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.131:51069)
0810 01:01:16.280737 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:16.280827 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260810 01:01:16.281198 5876 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 01:01:16.337390 5880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:16.336584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.129:41381)
0810 01:01:16.337057 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:16.337154 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":227,"threads_started":1}
I20260810 01:01:16.810160 5881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:16.809238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.130:59985)
0810 01:01:16.809761 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:16.809887 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":223,"threads_started":1}
I20260810 01:01:17.282768 5882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:17.281806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.131:39843)
0810 01:01:17.282312 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:17.282513 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":254,"threads_started":1}
I20260810 01:01:17.339112 5883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:17.338274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.129:36649)
0810 01:01:17.338864 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:17.338948 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":363,"threads_started":1}
I20260810 01:01:17.811825 5884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:17.811088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.130:38109)
0810 01:01:17.811567 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:17.811649 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":228,"threads_started":1}
I20260810 01:01:18.284562 5886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:18.283696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.131:53589)
0810 01:01:18.284249 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:18.284349 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":290,"threads_started":1}
I20260810 01:01:18.340876 5888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:18.340112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.129:40209)
0810 01:01:18.340610 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:18.340693 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":236,"threads_started":1}
I20260810 01:01:18.813527 5890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:18.812772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.130:59899)
0810 01:01:18.813243 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:18.813335 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":248,"threads_started":1}
I20260810 01:01:19.286284 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:19.285526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.131:60791)
0810 01:01:19.286022 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:19.286113 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":273,"threads_started":1}
I20260810 01:01:19.342700 5892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:19.341862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.129:46435)
0810 01:01:19.342325 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:19.342489 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":234,"threads_started":1}
I20260810 01:01:19.815187 5893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:19.814463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.130:43677)
0810 01:01:19.814904 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:19.814987 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":208,"threads_started":1}
I20260810 01:01:20.287940 5894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:20.287213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.131:54383)
0810 01:01:20.287686 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:20.287769 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":260,"threads_started":1}
I20260810 01:01:20.344551 5896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:20.343799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.129:52515)
0810 01:01:20.344280 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:20.344376 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":236,"threads_started":1}
I20260810 01:01:20.816912 5897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:20.816149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.130:39717)
0810 01:01:20.816645 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:20.816733 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":262,"threads_started":1}
I20260810 01:01:21.289495 5898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:21.288804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.131:57171)
0810 01:01:21.289231 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:21.289322 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":221,"threads_started":1}
I20260810 01:01:21.346223 5899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:21.345511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.129:45351)
0810 01:01:21.345954 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:21.346050 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":219,"threads_started":1}
I20260810 01:01:21.818655 5900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:21.817849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.130:34021)
0810 01:01:21.818331 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:21.818481 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":259,"threads_started":1}
I20260810 01:01:22.291167 5901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:22.290365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.131:49375)
0810 01:01:22.290909 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:22.291004 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":253,"threads_started":1}
I20260810 01:01:22.348037 5902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:22.347182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.129:58785)
0810 01:01:22.347709 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:22.347806 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":243,"threads_started":1}
I20260810 01:01:22.820569 5903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:22.819734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.130:60233)
0810 01:01:22.820233 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:22.820314 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":217,"threads_started":1}
I20260810 01:01:23.292996 5904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:23.292152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.131:55311)
0810 01:01:23.292673 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:23.292761 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":241,"threads_started":1}
I20260810 01:01:23.349730 5905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:23.348997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.129:57245)
0810 01:01:23.349473 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:23.349560 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":260,"threads_started":1}
I20260810 01:01:23.822604 5906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:23.821617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.130:56497)
0810 01:01:23.822221 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:23.822362 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":341,"threads_started":1}
I20260810 01:01:24.294802 5907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:24.293996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.131:43467)
0810 01:01:24.294521 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:24.294614 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":245,"threads_started":1}
I20260810 01:01:24.351527 5908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:24.350718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.129:59175)
0810 01:01:24.351221 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:24.351308 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":243,"threads_started":1}
I20260810 01:01:24.824295 5909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:24.823560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.130:48525)
0810 01:01:24.824030 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:24.824113 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":227,"threads_started":1}
I20260810 01:01:25.296661 5910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:25.295835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.131:42231)
0810 01:01:25.296368 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:25.296468 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":260,"threads_started":1}
I20260810 01:01:25.353308 5911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:25.352528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.129:42553)
0810 01:01:25.353027 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:25.353129 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":276,"threads_started":1}
I20260810 01:01:25.826071 5912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:25.825313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.130:51827)
0810 01:01:25.825813 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:25.825898 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":285,"threads_started":1}
I20260810 01:01:26.298446 5913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:26.297676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.131:38847)
0810 01:01:26.298134 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:26.298220 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":249,"threads_started":1}
I20260810 01:01:26.355274 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:26.354465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.129:57051)
0810 01:01:26.354964 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:26.355076 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":246,"threads_started":1}
I20260810 01:01:26.827858 5915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:26.827049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.130:41721)
0810 01:01:26.827548 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:26.827657 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":234,"threads_started":1}
I20260810 01:01:27.300168 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:27.299398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.131:60173)
0810 01:01:27.299887 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:27.299983 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":240,"threads_started":1}
I20260810 01:01:27.357060 5917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:27.356299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.129:33253)
0810 01:01:27.356791 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:27.356875 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":266,"threads_started":1}
I20260810 01:01:27.829614 5918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:27.828880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.130:60979)
0810 01:01:27.829338 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:27.829448 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":226,"threads_started":1}
I20260810 01:01:28.302000 5919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:28.301229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.131:35133)
0810 01:01:28.301723 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:28.301835 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":266,"threads_started":1}
I20260810 01:01:28.359079 5920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:28.358120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.129:32823)
0810 01:01:28.358710 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:28.358811 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":307,"threads_started":1}
I20260810 01:01:28.831554 5921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:28.830717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.130:37495)
0810 01:01:28.831221 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:28.831356 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":249,"threads_started":1}
I20260810 01:01:29.303980 5922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:29.303089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.131:38317)
0810 01:01:29.303640 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:29.303748 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":273,"threads_started":1}
I20260810 01:01:29.360908 5923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:29.360084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.129:44143)
0810 01:01:29.360597 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:29.360695 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":270,"threads_started":1}
I20260810 01:01:29.833246 5924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:29.832549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.130:50921)
0810 01:01:29.832978 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:29.833058 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":224,"threads_started":1}
I20260810 01:01:30.305754 5925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:30.304975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.131:56615)
0810 01:01:30.305458 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:30.305557 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":255,"threads_started":1}
I20260810 01:01:30.363046 5926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:30.362182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.129:37655)
0810 01:01:30.362742 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:30.362855 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":344,"threads_started":1}
I20260810 01:01:30.835456 5927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:30.834691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.130:52045)
0810 01:01:30.835152 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:30.835261 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":248,"threads_started":1}
I20260810 01:01:31.279918 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 5397
I20260810 01:01:31.304440 5928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:31.281202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50760 (local address 127.20.81.129:46663)
0810 01:01:31.281940 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:31.281958 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:31.282008 (+ 50us) server_negotiation.cc:407] Connection header received
0810 01:01:31.283949 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:31.283959 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:31.284418 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:31.284738 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:31.286147 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:31.287511 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:31.288809 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:31.289511 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:31.289685 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:31.297675 (+ 7990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:31.297683 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:31.297701 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:31.297758 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:31.300779 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:31.301535 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:31.301543 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:31.301550 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:31.301730 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:31.302169 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:31.302177 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:31.302183 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:31.302836 (+ 653us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:31.303179 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:31.303686 (+ 507us) server_negotiation.cc:299] Negotiation successful
0810 01:01:31.304135 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":302,"threads_started":1}
I20260810 01:01:31.305867 5576 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:31.308293 5931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:31.307226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.131:52115)
0810 01:01:31.307782 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:31.307932 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":289,"threads_started":1}
I20260810 01:01:31.395323 5933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:31.390698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:38513 (local address 127.20.81.129:37309)
0810 01:01:31.394874 (+ 4176us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:31.395070 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4057,"thread_start_us":3891,"threads_started":1}
I20260810 01:01:31.395651 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 5397
I20260810 01:01:31.417829 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 5584
I20260810 01:01:31.436568 5934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:31.419030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55706 (local address 127.20.81.130:35483)
0810 01:01:31.419625 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:31.419640 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:31.419677 (+ 37us) server_negotiation.cc:407] Connection header received
0810 01:01:31.421333 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:31.421343 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:31.421812 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:31.422107 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:31.423645 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:31.424922 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:31.426288 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:31.427039 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:31.427173 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:31.430035 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:31.430042 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:31.430058 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:31.430098 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:31.433211 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:31.434004 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:31.434011 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:31.434017 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:31.434190 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:31.434662 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:31.434665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:31.434667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:31.435223 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:31.435463 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:31.435922 (+ 459us) server_negotiation.cc:299] Negotiation successful
0810 01:01:31.436257 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":275,"threads_started":1}
I20260810 01:01:31.438094 5724 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:31.524619 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 5584
I20260810 01:01:31.546464 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 5732
I20260810 01:01:31.565297 5936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:31.547640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58252 (local address 127.20.81.131:39567)
0810 01:01:31.548273 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:31.548287 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:31.548328 (+ 41us) server_negotiation.cc:407] Connection header received
0810 01:01:31.550012 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:31.550022 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:31.550552 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:31.550863 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:31.552346 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:31.553618 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:31.555032 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:31.555746 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:31.555908 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:31.558821 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:31.558829 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:31.558845 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:31.558893 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:31.561892 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:31.562709 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:31.562718 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:31.562730 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:31.563006 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:31.563407 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:31.563412 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:31.563416 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:31.563967 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:31.564204 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:31.564635 (+ 431us) server_negotiation.cc:299] Negotiation successful
0810 01:01:31.564987 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":302,"threads_started":1}
I20260810 01:01:31.566879 5871 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:31.650920 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 5732
2026-08-10T01:01:31Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25098 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:01:31 dist-test-slave-gp6n krb5kdc[5940](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:01:31 dist-test-slave-gp6n krb5kdc[5940](info): set up 2 sockets
Aug 10 01:01:31 dist-test-slave-gp6n krb5kdc[5940](info): commencing operation
krb5kdc: starting...
W20260810 01:01:32.869832 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:01:33 dist-test-slave-gp6n krb5kdc[5940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:01:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:01:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 01:01:33.179312 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:44961
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:38185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:44961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:01:33.537437 5954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:33.537786 5954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:33.537947 5954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:33.545732 5954 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 01:01:33.545861 5954 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 01:01:33.545945 5954 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 01:01:33.546022 5954 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:01:33.546109 5954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:33.546214 5954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:33.546299 5954 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:01:33.546389 5954 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:01:33.559535 5954 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:38185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:44961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:44961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:33.560886 5954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:33.562245 5954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 01:01:33.575196 5954 server_base.cc:1034] running on GCE node
W20260810 01:01:33.575130 5961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:33.575083 5963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:33.575110 5959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:33.576267 5954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:33.577667 5954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:33.578958 5954 hybrid_clock.cc:648] HybridClock initialized: now 1786323693578884 us; error 38 us; skew 500 ppm
I20260810 01:01:33.579389 5954 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:33 dist-test-slave-gp6n krb5kdc[5940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323693, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:33.582948 5954 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 01:01:33.584586 5954 webserver.cc:460] Webserver started at http://127.20.81.190:42755/ using document root <none> and password file <none>
I20260810 01:01:33.585369 5954 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:33.585520 5954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:33.585873 5954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:01:33.589134 5954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "15a422ff679047df91331709f6a13913"
format_stamp: "Formatted at 2026-08-10 01:01:33 on dist-test-slave-gp6n"
server_key: "8517a733b5ee905f00fbbb9b83c4158c"
server_key_iv: "5040858cf93f24c30655b13fab6ebb3a"
server_key_version: "encryptionkey@0"
I20260810 01:01:33.590020 5954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "15a422ff679047df91331709f6a13913"
format_stamp: "Formatted at 2026-08-10 01:01:33 on dist-test-slave-gp6n"
server_key: "8517a733b5ee905f00fbbb9b83c4158c"
server_key_iv: "5040858cf93f24c30655b13fab6ebb3a"
server_key_version: "encryptionkey@0"
I20260810 01:01:33.595520 5954 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 01:01:33.599350 5971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:33.600857 5954 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 01:01:33.601007 5954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "15a422ff679047df91331709f6a13913"
format_stamp: "Formatted at 2026-08-10 01:01:33 on dist-test-slave-gp6n"
server_key: "8517a733b5ee905f00fbbb9b83c4158c"
server_key_iv: "5040858cf93f24c30655b13fab6ebb3a"
server_key_version: "encryptionkey@0"
I20260810 01:01:33.601203 5954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:33.614800 5954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:33.619381 5954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:33.619611 5954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:33.639014 5954 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:44961
I20260810 01:01:33.639048 6028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:44961 every 8 connection(s)
I20260810 01:01:33.640450 5954 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:01:33.644284 6029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:01:33.650103 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 5954
I20260810 01:01:33.650353 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:01:33.651237 20806 external_mini_cluster.cc:1428] Setting key af3d8d199fc4ba752ad191b1a9ee3fa6
I20260810 01:01:33.656221 6029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913: Bootstrap starting.
I20260810 01:01:33.660082 6029 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913: Neither blocks nor log segments found. Creating new log.
I20260810 01:01:33.661382 6029 log.cc:824] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913: Log is configured to *not* fsync() on all Append() calls
I20260810 01:01:33.664752 6029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913: No bootstrap required, opened a new log
Aug 10 01:01:33 dist-test-slave-gp6n krb5kdc[5940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:01:33.675429 6029 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a422ff679047df91331709f6a13913" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44961 } }
I20260810 01:01:33.675948 6029 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:01:33.676106 6029 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15a422ff679047df91331709f6a13913, State: Initialized, Role: FOLLOWER
I20260810 01:01:33.676805 6029 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a422ff679047df91331709f6a13913" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44961 } }
I20260810 01:01:33.677093 6029 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:01:33.677242 6029 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:01:33.677421 6029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:01:33.677769 6033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:33.653539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35360 (local address 127.20.81.190:44961)
0810 01:01:33.654487 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:33.654506 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:33.654551 (+ 45us) server_negotiation.cc:407] Connection header received
0810 01:01:33.656308 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:33.656330 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:33.656763 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:33.657102 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:33.659060 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:33.660495 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:33.662472 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:33.663275 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:33.663404 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:33.668381 (+ 4977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:33.668397 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:33.668419 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:33.668476 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:33.672803 (+ 4327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:33.673595 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:33.673605 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:33.673618 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:33.673869 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:33.674258 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:33.674264 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:33.674267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:33.675021 (+ 754us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:33.675327 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:33.675778 (+ 451us) server_negotiation.cc:299] Negotiation successful
0810 01:01:33.676276 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":436,"threads_started":1}
I20260810 01:01:33.680408 6029 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a422ff679047df91331709f6a13913" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44961 } }
I20260810 01:01:33.680994 6029 leader_election.cc:302] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15a422ff679047df91331709f6a13913; no voters:
I20260810 01:01:33.681505 6029 leader_election.cc:288] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:01:33.681701 6036 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:01:33.682427 6036 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 1 LEADER]: Becoming Leader. State: Replica: 15a422ff679047df91331709f6a13913, State: Running, Role: LEADER
I20260810 01:01:33.682998 6036 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a422ff679047df91331709f6a13913" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44961 } }
I20260810 01:01:33.683604 6029 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:01:33.685896 6038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15a422ff679047df91331709f6a13913. Latest consensus state: current_term: 1 leader_uuid: "15a422ff679047df91331709f6a13913" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a422ff679047df91331709f6a13913" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44961 } } }
I20260810 01:01:33.686375 6038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [sys.catalog]: This master's current role is: LEADER
I20260810 01:01:33.687464 6037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15a422ff679047df91331709f6a13913" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a422ff679047df91331709f6a13913" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44961 } } }
I20260810 01:01:33.687757 6037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [sys.catalog]: This master's current role is: LEADER
I20260810 01:01:33.691877 6041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:01:33.697327 6041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:01:33.707041 6041 catalog_manager.cc:1347] Generated new cluster ID: 7ce9c0c0e27d4831bef4f47cee0b273c
I20260810 01:01:33.707175 6041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:01:33.732494 6041 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:01:33.733986 6041 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:01:33.742980 6041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913: Generated new TSK 0
I20260810 01:01:33.743922 6041 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.81.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
I20260810 01:01:33.822760 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44961
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:38185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:01:34.175262 6059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:34.175557 6059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:34.175655 6059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:34.182993 6059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:34.183101 6059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:34.183216 6059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 01:01:34.193881 6059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:38185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:34.195245 6059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:34.196723 6059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:01:34.210904 6064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:34.210868 6067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:34.210947 6065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:34.211933 6059 server_base.cc:1034] running on GCE node
I20260810 01:01:34.212581 6059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:34.213627 6059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:34.214861 6059 hybrid_clock.cc:648] HybridClock initialized: now 1786323694214822 us; error 50 us; skew 500 ppm
I20260810 01:01:34.215202 6059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:34 dist-test-slave-gp6n krb5kdc[5940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323694, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:34.218856 6059 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 01:01:34.221042 6059 webserver.cc:460] Webserver started at http://127.20.81.129:45695/ using document root <none> and password file <none>
I20260810 01:01:34.222033 6059 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:34.222236 6059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:34.222676 6059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:01:34.227224 6059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "eec9553789ba459c869a9fa7b56a7302"
format_stamp: "Formatted at 2026-08-10 01:01:34 on dist-test-slave-gp6n"
server_key: "97459b84813e059360046487bc9538af"
server_key_iv: "c5fdf0988dc90e94e7d9caba44a13dec"
server_key_version: "encryptionkey@0"
I20260810 01:01:34.228075 6059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "eec9553789ba459c869a9fa7b56a7302"
format_stamp: "Formatted at 2026-08-10 01:01:34 on dist-test-slave-gp6n"
server_key: "97459b84813e059360046487bc9538af"
server_key_iv: "c5fdf0988dc90e94e7d9caba44a13dec"
server_key_version: "encryptionkey@0"
I20260810 01:01:34.233747 6059 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 01:01:34.237607 6074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:34.239184 6059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:01:34.239365 6059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "eec9553789ba459c869a9fa7b56a7302"
format_stamp: "Formatted at 2026-08-10 01:01:34 on dist-test-slave-gp6n"
server_key: "97459b84813e059360046487bc9538af"
server_key_iv: "c5fdf0988dc90e94e7d9caba44a13dec"
server_key_version: "encryptionkey@0"
I20260810 01:01:34.239548 6059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:34.252130 6059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:34.256709 6059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:34.257030 6059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:34.258034 6059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:01:34.259492 6059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:01:34.259568 6059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:34.259721 6059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:01:34.259786 6059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:34.295873 6059 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:33271
I20260810 01:01:34.295922 6199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:33271 every 8 connection(s)
I20260810 01:01:34.297241 6059 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 01:01:34.304133 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 6059
I20260810 01:01:34.304378 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 01:01:34.305328 20806 external_mini_cluster.cc:1428] Setting key bd6fb1aeab142fb94a2e4ead96bf1285
Aug 10 01:01:34 dist-test-slave-gp6n krb5kdc[5940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323694, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:01:34.321141 6202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:34.299990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:49601 (local address 127.20.81.190:44961)
0810 01:01:34.300455 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:34.300460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:34.301655 (+ 1195us) server_negotiation.cc:407] Connection header received
0810 01:01:34.304524 (+ 2869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:34.304529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:34.304600 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:34.304732 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:34.306924 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:34.307817 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:34.309558 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:34.309973 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:34.310116 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:34.313965 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:34.313970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:34.313974 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:34.314008 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:34.317460 (+ 3452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:34.318476 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:34.318484 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:34.318489 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:34.318629 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:34.319134 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:34.319141 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:34.319146 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:34.319485 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:34.319621 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:34.320510 (+ 889us) server_negotiation.cc:299] Negotiation successful
0810 01:01:34.320753 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":240,"threads_started":1}
I20260810 01:01:34.322151 6203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:34.300612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.129:49601)
0810 01:01:34.301458 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:34.301523 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:34.304174 (+ 2651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:34.305016 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:34.305029 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:34.305904 (+ 875us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:01:34.306733 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:34.306760 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:34.307980 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:34.307985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:34.309406 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:34.309419 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:34.310212 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:34.310222 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:34.310550 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:34.311275 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:01:34.311299 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:01:34.313732 (+ 2433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:01:34.317617 (+ 3885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:34.317629 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:34.317650 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:34.318325 (+ 675us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:34.318740 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:34.318747 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:34.318753 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:34.319014 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:34.319625 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:01:34.319637 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:01:34.320096 (+ 459us) client_negotiation.cc:769] Sending connection context
0810 01:01:34.320377 (+ 281us) client_negotiation.cc:240] Negotiation successful
0810 01:01:34.320930 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":306,"threads_started":1}
I20260810 01:01:34.323904 6200 heartbeater.cc:347] Connected to a master server at 127.20.81.190:44961
I20260810 01:01:34.324222 6200 heartbeater.cc:464] Registering TS with master...
I20260810 01:01:34.324991 6200 heartbeater.cc:511] Master 127.20.81.190:44961 requested a full tablet report, sending...
I20260810 01:01:34.327172 5989 ts_manager.cc:194] Registered new tserver with Master: eec9553789ba459c869a9fa7b56a7302 (127.20.81.129:33271)
I20260810 01:01:34.328612 5989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:49601
WARNING: no policy specified for kudu/127.20.81.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
I20260810 01:01:34.373735 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44961
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:38185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:01:34.736058 6206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:34.736389 6206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:34.736558 6206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:34.743937 6206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:34.744100 6206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:34.744308 6206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 01:01:34.754199 6206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:38185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:34.755612 6206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:34.756981 6206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:01:34.770210 6212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:34.770238 6214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:34.770166 6211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:34.770948 6206 server_base.cc:1034] running on GCE node
I20260810 01:01:34.771641 6206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:34.772578 6206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:34.773805 6206 hybrid_clock.cc:648] HybridClock initialized: now 1786323694773768 us; error 56 us; skew 500 ppm
I20260810 01:01:34.774128 6206 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:34 dist-test-slave-gp6n krb5kdc[5940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323694, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:34.777295 6206 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 01:01:34.778811 6206 webserver.cc:460] Webserver started at http://127.20.81.130:37045/ using document root <none> and password file <none>
I20260810 01:01:34.779495 6206 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:34.779640 6206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:34.779961 6206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:01:34.782959 6206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "24f57e6e4bc9455fa63a2770cf8c9a31"
format_stamp: "Formatted at 2026-08-10 01:01:34 on dist-test-slave-gp6n"
server_key: "f646330df02b2bce4cb8b12179c3191f"
server_key_iv: "d836eaafbc882efbf0d8f72f78e4efa1"
server_key_version: "encryptionkey@0"
I20260810 01:01:34.783716 6206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "24f57e6e4bc9455fa63a2770cf8c9a31"
format_stamp: "Formatted at 2026-08-10 01:01:34 on dist-test-slave-gp6n"
server_key: "f646330df02b2bce4cb8b12179c3191f"
server_key_iv: "d836eaafbc882efbf0d8f72f78e4efa1"
server_key_version: "encryptionkey@0"
I20260810 01:01:34.788823 6206 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 01:01:34.792410 6222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:34.793804 6206 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 01:01:34.793982 6206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "24f57e6e4bc9455fa63a2770cf8c9a31"
format_stamp: "Formatted at 2026-08-10 01:01:34 on dist-test-slave-gp6n"
server_key: "f646330df02b2bce4cb8b12179c3191f"
server_key_iv: "d836eaafbc882efbf0d8f72f78e4efa1"
server_key_version: "encryptionkey@0"
I20260810 01:01:34.794186 6206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:34.824780 6206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:34.829035 6206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:34.829344 6206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:34.830333 6206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:01:34.831739 6206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:01:34.831848 6206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:34.832002 6206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:01:34.832082 6206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:34.868569 6206 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:36451
I20260810 01:01:34.868628 6347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:36451 every 8 connection(s)
I20260810 01:01:34.869968 6206 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 01:01:34.873116 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 6206
I20260810 01:01:34.873324 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 01:01:34.873958 20806 external_mini_cluster.cc:1428] Setting key dc6c1927da0101e466929b0b53e93335
Aug 10 01:01:34 dist-test-slave-gp6n krb5kdc[5940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323694, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:01:34.892602 6350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:34.872515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:40263 (local address 127.20.81.190:44961)
0810 01:01:34.872984 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:34.872990 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:34.874250 (+ 1260us) server_negotiation.cc:407] Connection header received
0810 01:01:34.876569 (+ 2319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:34.876573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:34.876633 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:34.876723 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:34.878866 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:34.879610 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:34.882025 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:34.882509 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:34.882601 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:34.886732 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:34.886737 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:34.886740 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:34.886772 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:34.889445 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:34.890175 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:34.890185 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:34.890188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:34.890315 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:34.890791 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:34.890800 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:34.890802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:34.891071 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:34.891233 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:34.891970 (+ 737us) server_negotiation.cc:299] Negotiation successful
0810 01:01:34.892201 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":228,"threads_started":1}
I20260810 01:01:34.893656 6351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:34.873121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.130:40263)
0810 01:01:34.873974 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:34.874128 (+ 154us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:34.876273 (+ 2145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:34.876994 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:34.877007 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:34.877822 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:01:34.878685 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:34.878711 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:34.879766 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:34.879774 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:34.881886 (+ 2112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:34.881903 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:34.882721 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:34.882730 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:34.883084 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:34.883799 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:01:34.883834 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:01:34.886513 (+ 2679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:01:34.889595 (+ 3082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:34.889604 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:34.889619 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:34.890046 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:34.890457 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:34.890465 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:34.890469 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:34.890666 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:34.891201 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:01:34.891208 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:01:34.891556 (+ 348us) client_negotiation.cc:769] Sending connection context
0810 01:01:34.891863 (+ 307us) client_negotiation.cc:240] Negotiation successful
0810 01:01:34.892428 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":330,"threads_started":1}
I20260810 01:01:34.895213 6348 heartbeater.cc:347] Connected to a master server at 127.20.81.190:44961
I20260810 01:01:34.895555 6348 heartbeater.cc:464] Registering TS with master...
I20260810 01:01:34.896503 6348 heartbeater.cc:511] Master 127.20.81.190:44961 requested a full tablet report, sending...
I20260810 01:01:34.898176 5989 ts_manager.cc:194] Registered new tserver with Master: 24f57e6e4bc9455fa63a2770cf8c9a31 (127.20.81.130:36451)
I20260810 01:01:34.899109 5989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:40263
WARNING: no policy specified for kudu/127.20.81.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
I20260810 01:01:34.943853 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44961
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:38185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:01:35.298738 6354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:35.299049 6354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:35.299211 6354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:35.307888 6354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:35.308037 6354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:35.308209 6354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 01:01:35.318373 6354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:38185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:44961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:35.320094 6354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:35.321606 6354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 01:01:35.331734 6200 heartbeater.cc:503] Master 127.20.81.190:44961 was elected leader, sending a full tablet report...
W20260810 01:01:35.336172 6360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:35.336290 6354 server_base.cc:1034] running on GCE node
W20260810 01:01:35.336133 6362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:35.336175 6359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:35.337248 6354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:35.338207 6354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:35.339459 6354 hybrid_clock.cc:648] HybridClock initialized: now 1786323695339360 us; error 73 us; skew 500 ppm
I20260810 01:01:35.339783 6354 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:35 dist-test-slave-gp6n krb5kdc[5940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323695, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:35.343005 6354 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 01:01:35.344498 6354 webserver.cc:460] Webserver started at http://127.20.81.131:33563/ using document root <none> and password file <none>
I20260810 01:01:35.345285 6354 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:35.345439 6354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:35.345755 6354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:01:35.348786 6354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8"
format_stamp: "Formatted at 2026-08-10 01:01:35 on dist-test-slave-gp6n"
server_key: "526433a6d191504e14fda4d038284f63"
server_key_iv: "07c73ada98560d66a4ea79d24bcc8857"
server_key_version: "encryptionkey@0"
I20260810 01:01:35.349571 6354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8"
format_stamp: "Formatted at 2026-08-10 01:01:35 on dist-test-slave-gp6n"
server_key: "526433a6d191504e14fda4d038284f63"
server_key_iv: "07c73ada98560d66a4ea79d24bcc8857"
server_key_version: "encryptionkey@0"
I20260810 01:01:35.354954 6354 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260810 01:01:35.358574 6369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:35.359944 6354 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 01:01:35.360121 6354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8"
format_stamp: "Formatted at 2026-08-10 01:01:35 on dist-test-slave-gp6n"
server_key: "526433a6d191504e14fda4d038284f63"
server_key_iv: "07c73ada98560d66a4ea79d24bcc8857"
server_key_version: "encryptionkey@0"
I20260810 01:01:35.360319 6354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:35.375320 6354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:35.379348 6354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:35.379664 6354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:35.380614 6354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:01:35.381960 6354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:01:35.382061 6354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:35.382210 6354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:01:35.382287 6354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:35.417775 6354 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:36651
I20260810 01:01:35.417825 6501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:36651 every 8 connection(s)
I20260810 01:01:35.419209 6354 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 01:01:35.422470 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 6354
I20260810 01:01:35.422691 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 01:01:35.423166 20806 external_mini_cluster.cc:1428] Setting key 784e198cfbbb7a643ed78efa12026549
Aug 10 01:01:35 dist-test-slave-gp6n krb5kdc[5940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323695, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:01:35.440068 6504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:35.421874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:53363 (local address 127.20.81.190:44961)
0810 01:01:35.422480 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:35.422489 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:35.423714 (+ 1225us) server_negotiation.cc:407] Connection header received
0810 01:01:35.426144 (+ 2430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:35.426157 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:35.426268 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:35.426351 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:35.427934 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.428817 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.430315 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.430758 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.430852 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:35.434112 (+ 3260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:35.434117 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:35.434120 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:35.434162 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:35.436949 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:35.437767 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:35.437772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:35.437777 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:35.437940 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:35.438397 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:35.438425 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:35.438428 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:35.438774 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:35.438949 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:35.439611 (+ 662us) server_negotiation.cc:299] Negotiation successful
0810 01:01:35.439789 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":295,"threads_started":1}
I20260810 01:01:35.440986 6505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:35.422603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.131:53363)
0810 01:01:35.423511 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:35.423568 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:35.425798 (+ 2230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:35.426671 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:35.426680 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:35.427221 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:01:35.427785 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:35.427801 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.428961 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.428965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:35.430194 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:35.430206 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.430949 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.430954 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:35.431148 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:35.431754 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:01:35.431776 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:01:35.433933 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:01:35.437088 (+ 3155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:35.437099 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:35.437115 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:35.437646 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:35.438068 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:35.438076 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:35.438082 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:35.438278 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:35.438911 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:01:35.438920 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:01:35.439279 (+ 359us) client_negotiation.cc:769] Sending connection context
0810 01:01:35.439522 (+ 243us) client_negotiation.cc:240] Negotiation successful
0810 01:01:35.439903 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":377,"threads_started":1}
I20260810 01:01:35.442382 6502 heartbeater.cc:347] Connected to a master server at 127.20.81.190:44961
I20260810 01:01:35.442767 6502 heartbeater.cc:464] Registering TS with master...
I20260810 01:01:35.443590 6502 heartbeater.cc:511] Master 127.20.81.190:44961 requested a full tablet report, sending...
I20260810 01:01:35.445122 5989 ts_manager.cc:194] Registered new tserver with Master: d5f9461b4f1d4474a34671fbc4a7d7f8 (127.20.81.131:36651)
I20260810 01:01:35.445925 5989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:53363
I20260810 01:01:35.451062 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 01:01:35.467518 6504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:35.455533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35368 (local address 127.20.81.190:44961)
0810 01:01:35.455754 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:35.455763 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:35.456134 (+ 371us) server_negotiation.cc:407] Connection header received
0810 01:01:35.456264 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:35.456267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:35.456347 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:35.456444 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:35.457825 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.458530 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.459801 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.460196 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.460288 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:35.462633 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:35.462637 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:35.462641 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:35.462668 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:35.465226 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:35.465830 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:35.465834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:35.465836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:35.465944 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:35.466268 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:35.466271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:35.466272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:35.466586 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:35.466770 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:35.467029 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 01:01:35.467184 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 01:01:35.471689 5989 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35368:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 01:01:35.478540 5989 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 01:01:35.508049 6517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:35.490848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52428 (local address 127.20.81.130:36451)
0810 01:01:35.499235 (+ 8387us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:35.499244 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:35.499277 (+ 33us) server_negotiation.cc:407] Connection header received
0810 01:01:35.499394 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:35.499403 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:35.499708 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:35.499924 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:01:35.500941 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.502447 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.504938 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.506087 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.506248 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:35.506500 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:35.507521 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0810 01:01:35.507697 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8211,"thread_start_us":8106,"threads_started":1}
I20260810 01:01:35.508077 6522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:35.491679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:33271 (local address 127.0.0.1:42442)
0810 01:01:35.499401 (+ 7722us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:35.499419 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:35.499623 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:35.500496 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:35.500501 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:35.500540 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:01:35.501195 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:35.501205 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.502996 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.503001 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:35.504775 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:35.504789 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.507191 (+ 2402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.507198 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:35.507377 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:35.507408 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 01:01:35.507495 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 01:01:35.507590 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7599,"mutex_wait_us":35,"spinlock_wait_cycles":896,"thread_start_us":281,"threads_started":1}
I20260810 01:01:35.508085 6516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:35.490367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:36451 (local address 127.0.0.1:52428)
0810 01:01:35.491577 (+ 1210us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:35.491619 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:35.491777 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:35.499967 (+ 8190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:35.499976 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:35.500019 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:01:35.500729 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:35.500755 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.502561 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.502567 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:35.504782 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:35.504794 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.506922 (+ 2128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.506937 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:35.507120 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:35.507191 (+ 71us) client_negotiation.cc:769] Sending connection context
0810 01:01:35.507323 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 01:01:35.507525 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":934,"mutex_wait_us":124,"thread_start_us":765,"threads_started":1}
I20260810 01:01:35.508908 6518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:35.491015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42442 (local address 127.20.81.129:33271)
0810 01:01:35.498951 (+ 7936us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:35.498964 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:35.499521 (+ 557us) server_negotiation.cc:407] Connection header received
0810 01:01:35.499809 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:35.499822 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:35.500075 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:35.500236 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:01:35.501398 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.502854 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.504999 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.506548 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.506687 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:35.506937 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:35.508542 (+ 1605us) server_negotiation.cc:299] Negotiation successful
0810 01:01:35.508681 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7722,"thread_start_us":7554,"threads_started":1}
I20260810 01:01:35.509189 6519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:35.491459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:36651 (local address 127.0.0.1:55218)
0810 01:01:35.492356 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:35.492377 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:35.492539 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:35.500496 (+ 7957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:35.500501 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:35.500550 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:01:35.501173 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:35.501193 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.504293 (+ 3100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.504299 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:35.506450 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:35.506465 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.508270 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.508277 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:35.508594 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:35.508634 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 01:01:35.508702 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 01:01:35.508825 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"mutex_wait_us":28,"spinlock_wait_cycles":3328,"thread_start_us":7683,"threads_started":1}
I20260810 01:01:35.510335 6523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:35.492051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55218 (local address 127.20.81.131:36651)
0810 01:01:35.499922 (+ 7871us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:35.499931 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:35.499960 (+ 29us) server_negotiation.cc:407] Connection header received
0810 01:01:35.500046 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:35.500055 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:35.500287 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:35.500494 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:01:35.502532 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.504128 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.506677 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.508035 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.509574 (+ 1539us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:35.509797 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:35.509849 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 01:01:35.510011 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7664,"thread_start_us":7515,"threads_started":1}
I20260810 01:01:35.512454 6274 tablet_service.cc:1467] Processing CreateTablet for tablet 45de2d0a69ee4ccd9f86f6fdc76cefae (DEFAULT_TABLE table=test-table [id=fc28c148d99a4cf2906e26f898e22821]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:01:35.512606 6127 tablet_service.cc:1467] Processing CreateTablet for tablet 45de2d0a69ee4ccd9f86f6fdc76cefae (DEFAULT_TABLE table=test-table [id=fc28c148d99a4cf2906e26f898e22821]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:01:35.513893 6127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45de2d0a69ee4ccd9f86f6fdc76cefae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:01:35.513978 6274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45de2d0a69ee4ccd9f86f6fdc76cefae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:01:35.513711 6426 tablet_service.cc:1467] Processing CreateTablet for tablet 45de2d0a69ee4ccd9f86f6fdc76cefae (DEFAULT_TABLE table=test-table [id=fc28c148d99a4cf2906e26f898e22821]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:01:35.515161 6426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45de2d0a69ee4ccd9f86f6fdc76cefae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:01:35.525012 6524 tablet_bootstrap.cc:492] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302: Bootstrap starting.
I20260810 01:01:35.528225 6524 tablet_bootstrap.cc:654] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302: Neither blocks nor log segments found. Creating new log.
I20260810 01:01:35.529002 6525 tablet_bootstrap.cc:492] T 45de2d0a69ee4ccd9f86f6fdc76cefae P 24f57e6e4bc9455fa63a2770cf8c9a31: Bootstrap starting.
I20260810 01:01:35.530954 6524 log.cc:824] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302: Log is configured to *not* fsync() on all Append() calls
I20260810 01:01:35.532435 6526 tablet_bootstrap.cc:492] T 45de2d0a69ee4ccd9f86f6fdc76cefae P d5f9461b4f1d4474a34671fbc4a7d7f8: Bootstrap starting.
I20260810 01:01:35.532640 6525 tablet_bootstrap.cc:654] T 45de2d0a69ee4ccd9f86f6fdc76cefae P 24f57e6e4bc9455fa63a2770cf8c9a31: Neither blocks nor log segments found. Creating new log.
I20260810 01:01:35.533950 6524 tablet_bootstrap.cc:492] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302: No bootstrap required, opened a new log
I20260810 01:01:35.533881 6525 log.cc:824] T 45de2d0a69ee4ccd9f86f6fdc76cefae P 24f57e6e4bc9455fa63a2770cf8c9a31: Log is configured to *not* fsync() on all Append() calls
I20260810 01:01:35.534147 6524 ts_tablet_manager.cc:1397] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260810 01:01:35.535775 6526 tablet_bootstrap.cc:654] T 45de2d0a69ee4ccd9f86f6fdc76cefae P d5f9461b4f1d4474a34671fbc4a7d7f8: Neither blocks nor log segments found. Creating new log.
I20260810 01:01:35.536993 6525 tablet_bootstrap.cc:492] T 45de2d0a69ee4ccd9f86f6fdc76cefae P 24f57e6e4bc9455fa63a2770cf8c9a31: No bootstrap required, opened a new log
I20260810 01:01:35.536962 6526 log.cc:824] T 45de2d0a69ee4ccd9f86f6fdc76cefae P d5f9461b4f1d4474a34671fbc4a7d7f8: Log is configured to *not* fsync() on all Append() calls
I20260810 01:01:35.537220 6525 ts_tablet_manager.cc:1397] T 45de2d0a69ee4ccd9f86f6fdc76cefae P 24f57e6e4bc9455fa63a2770cf8c9a31: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260810 01:01:35.539856 6526 tablet_bootstrap.cc:492] T 45de2d0a69ee4ccd9f86f6fdc76cefae P d5f9461b4f1d4474a34671fbc4a7d7f8: No bootstrap required, opened a new log
I20260810 01:01:35.540095 6526 ts_tablet_manager.cc:1397] T 45de2d0a69ee4ccd9f86f6fdc76cefae P d5f9461b4f1d4474a34671fbc4a7d7f8: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 01:01:35.542104 6524 raft_consensus.cc:348] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } }
I20260810 01:01:35.542475 6524 raft_consensus.cc:374] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:01:35.542572 6524 raft_consensus.cc:729] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eec9553789ba459c869a9fa7b56a7302, State: Initialized, Role: FOLLOWER
I20260810 01:01:35.543179 6524 consensus_queue.cc:260] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } }
I20260810 01:01:35.544998 6524 ts_tablet_manager.cc:1428] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302: Time spent starting tablet: real 0.011s user 0.008s sys 0.003s
I20260810 01:01:35.547654 6525 raft_consensus.cc:348] T 45de2d0a69ee4ccd9f86f6fdc76cefae P 24f57e6e4bc9455fa63a2770cf8c9a31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } }
I20260810 01:01:35.548081 6525 raft_consensus.cc:374] T 45de2d0a69ee4ccd9f86f6fdc76cefae P 24f57e6e4bc9455fa63a2770cf8c9a31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:01:35.548177 6525 raft_consensus.cc:729] T 45de2d0a69ee4ccd9f86f6fdc76cefae P 24f57e6e4bc9455fa63a2770cf8c9a31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24f57e6e4bc9455fa63a2770cf8c9a31, State: Initialized, Role: FOLLOWER
I20260810 01:01:35.548727 6525 consensus_queue.cc:260] T 45de2d0a69ee4ccd9f86f6fdc76cefae P 24f57e6e4bc9455fa63a2770cf8c9a31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } }
W20260810 01:01:35.549690 6201 tablet.cc:2369] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:01:35.550014 6348 heartbeater.cc:503] Master 127.20.81.190:44961 was elected leader, sending a full tablet report...
I20260810 01:01:35.551414 6525 ts_tablet_manager.cc:1428] T 45de2d0a69ee4ccd9f86f6fdc76cefae P 24f57e6e4bc9455fa63a2770cf8c9a31: Time spent starting tablet: real 0.014s user 0.012s sys 0.002s
I20260810 01:01:35.551131 6526 raft_consensus.cc:348] T 45de2d0a69ee4ccd9f86f6fdc76cefae P d5f9461b4f1d4474a34671fbc4a7d7f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } }
I20260810 01:01:35.551577 6526 raft_consensus.cc:374] T 45de2d0a69ee4ccd9f86f6fdc76cefae P d5f9461b4f1d4474a34671fbc4a7d7f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:01:35.551684 6526 raft_consensus.cc:729] T 45de2d0a69ee4ccd9f86f6fdc76cefae P d5f9461b4f1d4474a34671fbc4a7d7f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5f9461b4f1d4474a34671fbc4a7d7f8, State: Initialized, Role: FOLLOWER
I20260810 01:01:35.552105 6526 consensus_queue.cc:260] T 45de2d0a69ee4ccd9f86f6fdc76cefae P d5f9461b4f1d4474a34671fbc4a7d7f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } }
I20260810 01:01:35.553102 6502 heartbeater.cc:503] Master 127.20.81.190:44961 was elected leader, sending a full tablet report...
I20260810 01:01:35.554975 6526 ts_tablet_manager.cc:1428] T 45de2d0a69ee4ccd9f86f6fdc76cefae P d5f9461b4f1d4474a34671fbc4a7d7f8: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
W20260810 01:01:35.622087 6349 tablet.cc:2369] T 45de2d0a69ee4ccd9f86f6fdc76cefae P 24f57e6e4bc9455fa63a2770cf8c9a31: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 01:01:35.670919 6503 tablet.cc:2369] T 45de2d0a69ee4ccd9f86f6fdc76cefae P d5f9461b4f1d4474a34671fbc4a7d7f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:01:35.760082 6531 raft_consensus.cc:482] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:01:35.760440 6531 raft_consensus.cc:504] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } }
I20260810 01:01:35.762212 6531 leader_election.cc:288] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24f57e6e4bc9455fa63a2770cf8c9a31 (127.20.81.130:36451), d5f9461b4f1d4474a34671fbc4a7d7f8 (127.20.81.131:36651)
I20260810 01:01:35.770089 6523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:35.762864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:39657 (local address 127.20.81.131:36651)
0810 01:01:35.763090 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:35.763098 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:35.763847 (+ 749us) server_negotiation.cc:407] Connection header received
0810 01:01:35.764061 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:35.764066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:35.764147 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:35.764250 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:01:35.765139 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.766284 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.768279 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.769042 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.769166 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:35.769266 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:35.769674 (+ 408us) server_negotiation.cc:299] Negotiation successful
0810 01:01:35.769811 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260810 01:01:35.770219 6534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:35.762670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:36651 (local address 127.20.81.129:39657)
0810 01:01:35.763724 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:35.763742 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:35.763900 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:35.764307 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:35.764313 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:35.764348 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:01:35.764961 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:35.764980 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.766456 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.766461 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:35.768108 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:35.768121 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.769262 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.769269 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:35.769480 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:35.769537 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 01:01:35.769611 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 01:01:35.769746 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":957,"mutex_wait_us":124,"spinlock_wait_cycles":26880,"thread_start_us":555,"threads_started":1}
I20260810 01:01:35.770262 6535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:35.762614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:36451 (local address 127.20.81.129:47665)
0810 01:01:35.763690 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:35.763716 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:35.763900 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:35.764309 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:35.764313 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:35.764351 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:01:35.764984 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:35.764993 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.766522 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.766526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:35.768115 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:35.768128 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.769391 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.769396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:35.769551 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:35.769574 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 01:01:35.769691 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 01:01:35.769813 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":922,"mutex_wait_us":104,"spinlock_wait_cycles":31488,"thread_start_us":656,"threads_started":1}
I20260810 01:01:35.770113 6517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:35.762761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:47665 (local address 127.20.81.130:36451)
0810 01:01:35.763007 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:35.763014 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:35.763836 (+ 822us) server_negotiation.cc:407] Connection header received
0810 01:01:35.764097 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:35.764103 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:35.764206 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:35.764296 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:01:35.765148 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.766321 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.768309 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.769126 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.769315 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:35.769397 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:35.769734 (+ 337us) server_negotiation.cc:299] Negotiation successful
0810 01:01:35.769871 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 01:01:35.771406 6450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45de2d0a69ee4ccd9f86f6fdc76cefae" candidate_uuid: "eec9553789ba459c869a9fa7b56a7302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" is_pre_election: true
I20260810 01:01:35.771405 6297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45de2d0a69ee4ccd9f86f6fdc76cefae" candidate_uuid: "eec9553789ba459c869a9fa7b56a7302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" is_pre_election: true
I20260810 01:01:35.772004 6297 raft_consensus.cc:2454] T 45de2d0a69ee4ccd9f86f6fdc76cefae P 24f57e6e4bc9455fa63a2770cf8c9a31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eec9553789ba459c869a9fa7b56a7302 in term 0.
I20260810 01:01:35.772004 6450 raft_consensus.cc:2454] T 45de2d0a69ee4ccd9f86f6fdc76cefae P d5f9461b4f1d4474a34671fbc4a7d7f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eec9553789ba459c869a9fa7b56a7302 in term 0.
I20260810 01:01:35.772759 6079 leader_election.cc:302] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 24f57e6e4bc9455fa63a2770cf8c9a31, d5f9461b4f1d4474a34671fbc4a7d7f8, eec9553789ba459c869a9fa7b56a7302; no voters:
I20260810 01:01:35.773212 6531 raft_consensus.cc:2781] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:01:35.773362 6531 raft_consensus.cc:482] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:01:35.773486 6531 raft_consensus.cc:3037] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:01:35.775775 6531 raft_consensus.cc:504] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } }
I20260810 01:01:35.776400 6531 leader_election.cc:288] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302 [CANDIDATE]: Term 1 election: Requested vote from peers 24f57e6e4bc9455fa63a2770cf8c9a31 (127.20.81.130:36451), d5f9461b4f1d4474a34671fbc4a7d7f8 (127.20.81.131:36651)
I20260810 01:01:35.776824 6297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45de2d0a69ee4ccd9f86f6fdc76cefae" candidate_uuid: "eec9553789ba459c869a9fa7b56a7302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31"
I20260810 01:01:35.776839 6450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45de2d0a69ee4ccd9f86f6fdc76cefae" candidate_uuid: "eec9553789ba459c869a9fa7b56a7302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8"
I20260810 01:01:35.777010 6297 raft_consensus.cc:3037] T 45de2d0a69ee4ccd9f86f6fdc76cefae P 24f57e6e4bc9455fa63a2770cf8c9a31 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:01:35.777040 6450 raft_consensus.cc:3037] T 45de2d0a69ee4ccd9f86f6fdc76cefae P d5f9461b4f1d4474a34671fbc4a7d7f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:01:35.779249 6450 raft_consensus.cc:2454] T 45de2d0a69ee4ccd9f86f6fdc76cefae P d5f9461b4f1d4474a34671fbc4a7d7f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eec9553789ba459c869a9fa7b56a7302 in term 1.
I20260810 01:01:35.779253 6297 raft_consensus.cc:2454] T 45de2d0a69ee4ccd9f86f6fdc76cefae P 24f57e6e4bc9455fa63a2770cf8c9a31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eec9553789ba459c869a9fa7b56a7302 in term 1.
I20260810 01:01:35.779688 6079 leader_election.cc:302] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5f9461b4f1d4474a34671fbc4a7d7f8, eec9553789ba459c869a9fa7b56a7302; no voters:
I20260810 01:01:35.780045 6531 raft_consensus.cc:2781] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:01:35.780465 6531 raft_consensus.cc:686] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302 [term 1 LEADER]: Becoming Leader. State: Replica: eec9553789ba459c869a9fa7b56a7302, State: Running, Role: LEADER
I20260810 01:01:35.781025 6531 consensus_queue.cc:237] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } }
I20260810 01:01:35.785665 5988 catalog_manager.cc:5495] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302 reported cstate change: term changed from 0 to 1, leader changed from <none> to eec9553789ba459c869a9fa7b56a7302 (127.20.81.129). New cstate: current_term: 1 leader_uuid: "eec9553789ba459c869a9fa7b56a7302" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } health_report { overall_health: UNKNOWN } } }
I20260810 01:01:35.868517 6518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:35.862799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42456 (local address 127.20.81.129:33271)
0810 01:01:35.863038 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:35.863044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:35.863062 (+ 18us) server_negotiation.cc:407] Connection header received
0810 01:01:35.863304 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:35.863308 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:35.863408 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:35.863588 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:01:35.864001 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.864758 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.866241 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.866677 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.866814 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:35.867327 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:01:35.867860 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:01:35.868033 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:35.868156 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 01:01:35.868242 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 01:01:35.877434 6450 raft_consensus.cc:1261] T 45de2d0a69ee4ccd9f86f6fdc76cefae P d5f9461b4f1d4474a34671fbc4a7d7f8 [term 1 FOLLOWER]: Refusing update from remote peer eec9553789ba459c869a9fa7b56a7302: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:01:35.877439 6297 raft_consensus.cc:1261] T 45de2d0a69ee4ccd9f86f6fdc76cefae P 24f57e6e4bc9455fa63a2770cf8c9a31 [term 1 FOLLOWER]: Refusing update from remote peer eec9553789ba459c869a9fa7b56a7302: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:01:35.878486 6531 consensus_queue.cc:1035] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:01:35.878821 6536 consensus_queue.cc:1035] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:01:35.907131 5987 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:01:35.907436 5987 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:01:35.911312 5987 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913: Sending DeleteTablet for 3 replicas of tablet 45de2d0a69ee4ccd9f86f6fdc76cefae
I20260810 01:01:35.912518 6127 tablet_service.cc:1514] Processing DeleteTablet for tablet 45de2d0a69ee4ccd9f86f6fdc76cefae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:01:35 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:42442
I20260810 01:01:35.912529 6274 tablet_service.cc:1514] Processing DeleteTablet for tablet 45de2d0a69ee4ccd9f86f6fdc76cefae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:01:35 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:52428
I20260810 01:01:35.912626 6426 tablet_service.cc:1514] Processing DeleteTablet for tablet 45de2d0a69ee4ccd9f86f6fdc76cefae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:01:35 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:55218
I20260810 01:01:35.913832 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 6059
I20260810 01:01:35.923002 6550 tablet_replica.cc:330] T 45de2d0a69ee4ccd9f86f6fdc76cefae P d5f9461b4f1d4474a34671fbc4a7d7f8: stopping tablet replica
I20260810 01:01:35.923519 6550 raft_consensus.cc:2229] T 45de2d0a69ee4ccd9f86f6fdc76cefae P d5f9461b4f1d4474a34671fbc4a7d7f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:01:35.924177 6550 raft_consensus.cc:2258] T 45de2d0a69ee4ccd9f86f6fdc76cefae P d5f9461b4f1d4474a34671fbc4a7d7f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:01:35.926183 6551 tablet_replica.cc:330] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302: stopping tablet replica
I20260810 01:01:35.926710 6551 raft_consensus.cc:2229] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302 [term 1 LEADER]: Raft consensus shutting down.
I20260810 01:01:35.927547 6551 raft_consensus.cc:2258] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:01:35.928413 6549 tablet_replica.cc:330] T 45de2d0a69ee4ccd9f86f6fdc76cefae P 24f57e6e4bc9455fa63a2770cf8c9a31: stopping tablet replica
I20260810 01:01:35.928882 6549 raft_consensus.cc:2229] T 45de2d0a69ee4ccd9f86f6fdc76cefae P 24f57e6e4bc9455fa63a2770cf8c9a31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:01:35.929611 6549 raft_consensus.cc:2258] T 45de2d0a69ee4ccd9f86f6fdc76cefae P 24f57e6e4bc9455fa63a2770cf8c9a31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:01:35.929836 6551 ts_tablet_manager.cc:1905] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:01:35.929828 6550 ts_tablet_manager.cc:1905] T 45de2d0a69ee4ccd9f86f6fdc76cefae P d5f9461b4f1d4474a34671fbc4a7d7f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:01:35.936254 6549 ts_tablet_manager.cc:1905] T 45de2d0a69ee4ccd9f86f6fdc76cefae P 24f57e6e4bc9455fa63a2770cf8c9a31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:01:35.937494 6551 ts_tablet_manager.cc:1918] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:01:35.937690 6551 log.cc:1196] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/wal/wals/45de2d0a69ee4ccd9f86f6fdc76cefae
I20260810 01:01:35.938009 6550 ts_tablet_manager.cc:1918] T 45de2d0a69ee4ccd9f86f6fdc76cefae P d5f9461b4f1d4474a34671fbc4a7d7f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:01:35.938220 6550 log.cc:1196] T 45de2d0a69ee4ccd9f86f6fdc76cefae P d5f9461b4f1d4474a34671fbc4a7d7f8: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/45de2d0a69ee4ccd9f86f6fdc76cefae
I20260810 01:01:35.938261 6551 ts_tablet_manager.cc:1939] T 45de2d0a69ee4ccd9f86f6fdc76cefae P eec9553789ba459c869a9fa7b56a7302: Deleting consensus metadata
I20260810 01:01:35.938771 6550 ts_tablet_manager.cc:1939] T 45de2d0a69ee4ccd9f86f6fdc76cefae P d5f9461b4f1d4474a34671fbc4a7d7f8: Deleting consensus metadata
I20260810 01:01:35.940199 5973 catalog_manager.cc:4841] TS eec9553789ba459c869a9fa7b56a7302 (127.20.81.129:33271): tablet 45de2d0a69ee4ccd9f86f6fdc76cefae (table test-table [id=fc28c148d99a4cf2906e26f898e22821]) successfully deleted
Aug 10 01:01:35 dist-test-slave-gp6n krb5kdc[5940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.129@KRBTEST.COM
I20260810 01:01:35.941169 5975 catalog_manager.cc:4841] TS d5f9461b4f1d4474a34671fbc4a7d7f8 (127.20.81.131:36651): tablet 45de2d0a69ee4ccd9f86f6fdc76cefae (table test-table [id=fc28c148d99a4cf2906e26f898e22821]) successfully deleted
I20260810 01:01:35.943408 6549 ts_tablet_manager.cc:1918] T 45de2d0a69ee4ccd9f86f6fdc76cefae P 24f57e6e4bc9455fa63a2770cf8c9a31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:01:35.943630 6549 log.cc:1196] T 45de2d0a69ee4ccd9f86f6fdc76cefae P 24f57e6e4bc9455fa63a2770cf8c9a31: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/45de2d0a69ee4ccd9f86f6fdc76cefae
I20260810 01:01:35.944137 6549 ts_tablet_manager.cc:1939] T 45de2d0a69ee4ccd9f86f6fdc76cefae P 24f57e6e4bc9455fa63a2770cf8c9a31: Deleting consensus metadata
I20260810 01:01:35.945776 5972 catalog_manager.cc:4841] TS 24f57e6e4bc9455fa63a2770cf8c9a31 (127.20.81.130:36451): tablet 45de2d0a69ee4ccd9f86f6fdc76cefae (table test-table [id=fc28c148d99a4cf2906e26f898e22821]) successfully deleted
I20260810 01:01:35.950182 6518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:35.915288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42464 (local address 127.20.81.129:33271)
0810 01:01:35.915517 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:35.915527 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:35.925496 (+ 9969us) server_negotiation.cc:407] Connection header received
0810 01:01:35.925740 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:35.925749 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:35.925894 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:35.926036 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:35.931077 (+ 5041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.932341 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.934580 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:35.935331 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:35.935441 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:35.942748 (+ 7307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:35.942763 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:35.942782 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:35.942838 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:35.947045 (+ 4207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:35.947730 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:35.947739 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:35.947743 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:35.947945 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:35.948322 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:35.948328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:35.948331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:35.948781 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:35.949095 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:35.949350 (+ 255us) server_negotiation.cc:299] Negotiation successful
0810 01:01:35.949795 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 01:01:35.950902 6193 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:36.050993 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 6059
I20260810 01:01:36.075268 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 6206
Aug 10 01:01:36 dist-test-slave-gp6n krb5kdc[5940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.130@KRBTEST.COM
I20260810 01:01:36.092141 6517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:36.076053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52444 (local address 127.20.81.130:36451)
0810 01:01:36.076242 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:36.076247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:36.076266 (+ 19us) server_negotiation.cc:407] Connection header received
0810 01:01:36.076419 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:36.076423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:36.076490 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:36.076577 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:36.078058 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:36.078888 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:36.080232 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:36.080622 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:36.080713 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:36.085666 (+ 4953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:36.085682 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:36.085698 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:36.085739 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:36.088991 (+ 3252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:36.089862 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:36.089870 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:36.089872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:36.090024 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:36.090473 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:36.090477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:36.090479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:36.090902 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:36.091124 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:36.091528 (+ 404us) server_negotiation.cc:299] Negotiation successful
0810 01:01:36.091765 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 01:01:36.092813 6343 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:36.183492 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 6206
I20260810 01:01:36.205349 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 6354
Aug 10 01:01:36 dist-test-slave-gp6n krb5kdc[5940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.131@KRBTEST.COM
I20260810 01:01:36.223323 6523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:36.206235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55232 (local address 127.20.81.131:36651)
0810 01:01:36.206691 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:36.206696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:36.206715 (+ 19us) server_negotiation.cc:407] Connection header received
0810 01:01:36.206776 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:36.206781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:36.206852 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:36.206949 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:36.208296 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:36.209128 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:36.210505 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:36.210932 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:36.211039 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:36.215682 (+ 4643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:36.215699 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:36.215715 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:36.215759 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:36.219312 (+ 3553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:36.220427 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:36.220438 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:36.220442 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:36.220680 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:36.221248 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:36.221255 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:36.221261 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:36.221790 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:36.222100 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:36.222662 (+ 562us) server_negotiation.cc:299] Negotiation successful
0810 01:01:36.222909 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260810 01:01:36.224171 6496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:36.318830 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 6354
I20260810 01:01:36.340188 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 5954
I20260810 01:01:36.341425 6024 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:36.454057 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 5954
I20260810 01:01:56.480430 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:44961
--webserver_interface=127.20.81.190
--webserver_port=42755
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:38185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:44961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:01:56.835633 6557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:56.836001 6557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:56.836163 6557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:56.843582 6557 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 01:01:56.843722 6557 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 01:01:56.843817 6557 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 01:01:56.843914 6557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:01:56.844019 6557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:56.844147 6557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:56.844236 6557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:01:56.844318 6557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:01:56.854853 6557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:38185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:44961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:44961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=42755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:56.856226 6557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:56.857627 6557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:01:56.888123 6565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:56.888084 6564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:56.888056 6567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:56.889850 6557 server_base.cc:1034] running on GCE node
I20260810 01:01:56.891085 6557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:56.892773 6557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:56.894027 6557 hybrid_clock.cc:648] HybridClock initialized: now 1786323716893947 us; error 55 us; skew 500 ppm
I20260810 01:01:56.894508 6557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:56 dist-test-slave-gp6n krb5kdc[5940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323716, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:56.898763 6557 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 01:01:56.904006 6557 webserver.cc:460] Webserver started at http://127.20.81.190:42755/ using document root <none> and password file <none>
I20260810 01:01:56.905059 6557 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:56.905257 6557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:56.912384 6557 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 01:01:56.915493 6577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:56.916934 6557 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 01:01:56.917127 6557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "15a422ff679047df91331709f6a13913"
format_stamp: "Formatted at 2026-08-10 01:01:33 on dist-test-slave-gp6n"
server_key: "8517a733b5ee905f00fbbb9b83c4158c"
server_key_iv: "5040858cf93f24c30655b13fab6ebb3a"
server_key_version: "encryptionkey@0"
I20260810 01:01:56.918236 6557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:56.953197 6557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:56.957949 6557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:56.958289 6557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:56.979728 6557 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:44961
I20260810 01:01:56.979754 6633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:44961 every 8 connection(s)
I20260810 01:01:56.981494 6557 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:01:56.989666 6634 sys_catalog.cc:263] Verifying existing consensus state
I20260810 01:01:56.991984 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 6557
I20260810 01:01:56.993124 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:33271
--local_ip_for_outbound_sockets=127.20.81.129
--tserver_master_addrs=127.20.81.190:44961
--webserver_port=45695
--webserver_interface=127.20.81.129
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:38185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 01:01:56.994221 6634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913: Bootstrap starting.
I20260810 01:01:57.019148 6634 log.cc:824] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913: Log is configured to *not* fsync() on all Append() calls
I20260810 01:01:57.032508 6634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 01:01:57.033118 6634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913: Bootstrap complete.
I20260810 01:01:57.042765 6634 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a422ff679047df91331709f6a13913" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44961 } }
I20260810 01:01:57.043637 6634 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15a422ff679047df91331709f6a13913, State: Initialized, Role: FOLLOWER
I20260810 01:01:57.044294 6634 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a422ff679047df91331709f6a13913" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44961 } }
I20260810 01:01:57.044550 6634 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:01:57.044675 6634 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:01:57.044886 6634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 1 FOLLOWER]: Advancing to term 2
I20260810 01:01:57.048873 6634 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a422ff679047df91331709f6a13913" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44961 } }
I20260810 01:01:57.049463 6634 leader_election.cc:302] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15a422ff679047df91331709f6a13913; no voters:
I20260810 01:01:57.050165 6634 leader_election.cc:288] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 01:01:57.050316 6638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 01:01:57.051452 6638 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 2 LEADER]: Becoming Leader. State: Replica: 15a422ff679047df91331709f6a13913, State: Running, Role: LEADER
I20260810 01:01:57.052099 6638 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a422ff679047df91331709f6a13913" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44961 } }
I20260810 01:01:57.052541 6634 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:01:57.055392 6641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15a422ff679047df91331709f6a13913. Latest consensus state: current_term: 2 leader_uuid: "15a422ff679047df91331709f6a13913" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a422ff679047df91331709f6a13913" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44961 } } }
I20260810 01:01:57.056003 6641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [sys.catalog]: This master's current role is: LEADER
I20260810 01:01:57.055392 6639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "15a422ff679047df91331709f6a13913" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a422ff679047df91331709f6a13913" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44961 } } }
I20260810 01:01:57.056633 6639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [sys.catalog]: This master's current role is: LEADER
I20260810 01:01:57.058872 6645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:01:57.067183 6645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:01:57.068641 6645 catalog_manager.cc:1259] Loaded cluster ID: 7ce9c0c0e27d4831bef4f47cee0b273c
I20260810 01:01:57.068796 6645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:01:57.072839 6645 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:01:57.074469 6645 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913: Loaded TSK: 0
I20260810 01:01:57.086895 6659 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 01:01:57.109452 6645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913: Generated new TSK 1
I20260810 01:01:57.109910 6645 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 01:01:57.363696 6636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:57.364035 6636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:57.364184 6636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:57.371825 6636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:57.371995 6636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:57.372169 6636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 01:01:57.382194 6636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:38185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:33271
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=45695
--tserver_master_addrs=127.20.81.190:44961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:57.383642 6636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:57.385565 6636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:01:57.401489 6665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:57.401512 6666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:57.402309 6668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:57.402729 6636 server_base.cc:1034] running on GCE node
I20260810 01:01:57.403385 6636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:57.404361 6636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:57.405598 6636 hybrid_clock.cc:648] HybridClock initialized: now 1786323717405544 us; error 50 us; skew 500 ppm
I20260810 01:01:57.406009 6636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:57 dist-test-slave-gp6n krb5kdc[5940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323717, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:57.409359 6636 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 01:01:57.410997 6636 webserver.cc:460] Webserver started at http://127.20.81.129:45695/ using document root <none> and password file <none>
I20260810 01:01:57.411701 6636 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:57.411844 6636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:57.417479 6636 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260810 01:01:57.420496 6676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:57.422043 6636 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 01:01:57.422221 6636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "eec9553789ba459c869a9fa7b56a7302"
format_stamp: "Formatted at 2026-08-10 01:01:34 on dist-test-slave-gp6n"
server_key: "97459b84813e059360046487bc9538af"
server_key_iv: "c5fdf0988dc90e94e7d9caba44a13dec"
server_key_version: "encryptionkey@0"
I20260810 01:01:57.423484 6636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:57.448123 6636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:57.452585 6636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:57.452939 6636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:57.453974 6636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:01:57.455370 6636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:01:57.455505 6636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:57.455639 6636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:01:57.455726 6636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:57.490557 6636 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:33271
I20260810 01:01:57.490612 6807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:33271 every 8 connection(s)
I20260810 01:01:57.492074 6636 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 01:01:57.497781 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 6636
I20260810 01:01:57.498571 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:36451
--local_ip_for_outbound_sockets=127.20.81.130
--tserver_master_addrs=127.20.81.190:44961
--webserver_port=37045
--webserver_interface=127.20.81.130
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:38185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 01:01:57 dist-test-slave-gp6n krb5kdc[5940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323717, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:01:57.518203 6810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:57.495234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:59653 (local address 127.20.81.190:44961)
0810 01:01:57.495833 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:57.495845 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:57.496538 (+ 693us) server_negotiation.cc:407] Connection header received
0810 01:01:57.498863 (+ 2325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:57.498902 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:57.499371 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:57.499790 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:57.501197 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:57.503227 (+ 2030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:57.504835 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:57.506006 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:57.506214 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:57.510095 (+ 3881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:57.510108 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:57.510129 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:57.510189 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:57.513530 (+ 3341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:57.514542 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:57.514549 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:57.514555 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:57.514717 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:57.515253 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:57.515256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:57.515258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:57.515721 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:57.515911 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:57.516888 (+ 977us) server_negotiation.cc:299] Negotiation successful
0810 01:01:57.517225 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":261,"threads_started":1}
I20260810 01:01:57.518784 6811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:57.495422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.129:59653)
0810 01:01:57.496352 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:57.496408 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:57.498247 (+ 1839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:57.499895 (+ 1648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:57.499902 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:57.500422 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:01:57.500982 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:57.500999 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:57.503415 (+ 2416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:57.503419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:57.504684 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:57.504697 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:57.506235 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:57.506239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:57.506508 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:57.507262 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:01:57.507291 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:01:57.509787 (+ 2496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:01:57.513690 (+ 3903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:57.513702 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:57.513721 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:57.514383 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:57.514850 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:57.514858 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:57.514864 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:57.515145 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:57.515879 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:01:57.515890 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:01:57.516377 (+ 487us) client_negotiation.cc:769] Sending connection context
0810 01:01:57.516713 (+ 336us) client_negotiation.cc:240] Negotiation successful
0810 01:01:57.517313 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":387,"threads_started":1}
I20260810 01:01:57.521867 6808 heartbeater.cc:347] Connected to a master server at 127.20.81.190:44961
I20260810 01:01:57.522249 6808 heartbeater.cc:464] Registering TS with master...
I20260810 01:01:57.523068 6808 heartbeater.cc:511] Master 127.20.81.190:44961 requested a full tablet report, sending...
I20260810 01:01:57.525532 6594 ts_manager.cc:194] Registered new tserver with Master: eec9553789ba459c869a9fa7b56a7302 (127.20.81.129:33271)
I20260810 01:01:57.527917 6594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:59653
W20260810 01:01:57.845319 6812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:57.845610 6812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:57.845710 6812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:57.853163 6812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:57.853271 6812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:57.853384 6812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 01:01:57.863540 6812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:38185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:36451
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=37045
--tserver_master_addrs=127.20.81.190:44961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:57.864820 6812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:57.866108 6812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:01:57.879655 6817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:57.879622 6821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:57.879810 6819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:57.880950 6812 server_base.cc:1034] running on GCE node
I20260810 01:01:57.881654 6812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:57.882671 6812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:57.883919 6812 hybrid_clock.cc:648] HybridClock initialized: now 1786323717883868 us; error 40 us; skew 500 ppm
I20260810 01:01:57.884281 6812 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:57 dist-test-slave-gp6n krb5kdc[5940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323717, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:57.887874 6812 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 01:01:57.889531 6812 webserver.cc:460] Webserver started at http://127.20.81.130:37045/ using document root <none> and password file <none>
I20260810 01:01:57.890285 6812 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:57.890508 6812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:57.896317 6812 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260810 01:01:57.899231 6829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:57.900646 6812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:01:57.900846 6812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "24f57e6e4bc9455fa63a2770cf8c9a31"
format_stamp: "Formatted at 2026-08-10 01:01:34 on dist-test-slave-gp6n"
server_key: "f646330df02b2bce4cb8b12179c3191f"
server_key_iv: "d836eaafbc882efbf0d8f72f78e4efa1"
server_key_version: "encryptionkey@0"
I20260810 01:01:57.901922 6812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:57.915642 6812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:57.920029 6812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:57.920274 6812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:57.921265 6812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:01:57.922791 6812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:01:57.922895 6812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:57.922983 6812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:01:57.923045 6812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:57.959962 6812 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:36451
I20260810 01:01:57.960049 6954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:36451 every 8 connection(s)
I20260810 01:01:57.961474 6812 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 01:01:57.964969 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 6812
I20260810 01:01:57.965914 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:36651
--local_ip_for_outbound_sockets=127.20.81.131
--tserver_master_addrs=127.20.81.190:44961
--webserver_port=33563
--webserver_interface=127.20.81.131
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:38185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 01:01:57 dist-test-slave-gp6n krb5kdc[5940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323717, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:01:57.988481 6810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:57.964406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:47995 (local address 127.20.81.190:44961)
0810 01:01:57.964678 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:57.964685 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:57.966068 (+ 1383us) server_negotiation.cc:407] Connection header received
0810 01:01:57.969304 (+ 3236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:57.969308 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:57.969389 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:57.969482 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:57.971886 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:57.972618 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:57.975441 (+ 2823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:57.975901 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:57.976014 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:57.980755 (+ 4741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:57.980762 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:57.980768 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:57.980816 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:57.984558 (+ 3742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:57.985532 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:57.985539 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:57.985543 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:57.985695 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:57.986272 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:57.986280 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:57.986283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:57.986738 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:57.986919 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:57.987815 (+ 896us) server_negotiation.cc:299] Negotiation successful
0810 01:01:57.988090 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260810 01:01:57.989746 6957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:57.964884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.130:47995)
0810 01:01:57.965830 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:57.965891 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:57.968957 (+ 3066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:57.969789 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:57.969805 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:57.970763 (+ 958us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:01:57.971670 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:57.971702 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:57.972798 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:57.972807 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:57.975259 (+ 2452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:57.975279 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:57.976158 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:57.976168 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:57.976534 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:57.977346 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:01:57.977384 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:01:57.980463 (+ 3079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:01:57.984727 (+ 4264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:57.984740 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:57.984758 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:57.985389 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:57.985834 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:57.985840 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:57.985845 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:57.986096 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:57.986898 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:01:57.986908 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:01:57.987366 (+ 458us) client_negotiation.cc:769] Sending connection context
0810 01:01:57.987672 (+ 306us) client_negotiation.cc:240] Negotiation successful
0810 01:01:57.988287 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":370,"threads_started":1}
I20260810 01:01:57.991595 6955 heartbeater.cc:347] Connected to a master server at 127.20.81.190:44961
I20260810 01:01:57.991954 6955 heartbeater.cc:464] Registering TS with master...
I20260810 01:01:57.992813 6955 heartbeater.cc:511] Master 127.20.81.190:44961 requested a full tablet report, sending...
I20260810 01:01:57.994690 6594 ts_manager.cc:194] Registered new tserver with Master: 24f57e6e4bc9455fa63a2770cf8c9a31 (127.20.81.130:36451)
I20260810 01:01:57.995981 6594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:47995
W20260810 01:01:58.322225 6958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:01:58.322674 6958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:01:58.322858 6958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:01:58.330345 6958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:01:58.330541 6958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:01:58.330721 6958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 01:01:58.340556 6958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:38185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:36651
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=33563
--tserver_master_addrs=127.20.81.190:44961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:01:58.341928 6958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:01:58.343358 6958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:01:58.357054 6964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:58.357084 6965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:01:58.357014 6967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:01:58.360024 6958 server_base.cc:1034] running on GCE node
I20260810 01:01:58.360801 6958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:01:58.361846 6958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:01:58.363107 6958 hybrid_clock.cc:648] HybridClock initialized: now 1786323718363054 us; error 48 us; skew 500 ppm
I20260810 01:01:58.363471 6958 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:01:58 dist-test-slave-gp6n krb5kdc[5940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323718, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:01:58.366935 6958 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 01:01:58.368534 6958 webserver.cc:460] Webserver started at http://127.20.81.131:33563/ using document root <none> and password file <none>
I20260810 01:01:58.369323 6958 fs_manager.cc:359] Metadata directory not provided
I20260810 01:01:58.369491 6958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:01:58.375300 6958 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 01:01:58.378042 6974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:58.379529 6958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 01:01:58.379724 6958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8"
format_stamp: "Formatted at 2026-08-10 01:01:35 on dist-test-slave-gp6n"
server_key: "526433a6d191504e14fda4d038284f63"
server_key_iv: "07c73ada98560d66a4ea79d24bcc8857"
server_key_version: "encryptionkey@0"
I20260810 01:01:58.380764 6958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:01:58.400110 6958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:01:58.404693 6958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:01:58.404994 6958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:01:58.406180 6958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:01:58.407728 6958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:01:58.407874 6958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:58.408001 6958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:01:58.408098 6958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:01:58.443873 6958 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:36651
I20260810 01:01:58.443961 7096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:36651 every 8 connection(s)
I20260810 01:01:58.445375 6958 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 01:01:58.454640 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 6958
Aug 10 01:01:58 dist-test-slave-gp6n krb5kdc[5940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323718, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:01:58.469204 6810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:58.448002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:42907 (local address 127.20.81.190:44961)
0810 01:01:58.448194 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:58.448199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:58.449389 (+ 1190us) server_negotiation.cc:407] Connection header received
0810 01:01:58.451324 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:58.451329 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:58.451425 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:58.451538 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:58.453181 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.454015 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.456389 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.456979 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.457142 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:58.461263 (+ 4121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:58.461272 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:58.461276 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:58.461476 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:58.465583 (+ 4107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:58.466390 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:58.466396 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:58.466435 (+ 39us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:58.466597 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:58.467147 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:58.467152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:58.467155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:58.467568 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:58.467731 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:58.468541 (+ 810us) server_negotiation.cc:299] Negotiation successful
0810 01:01:58.468775 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":71,"spinlock_wait_cycles":6912}
I20260810 01:01:58.469931 7099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:58.448498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.131:42907)
0810 01:01:58.449223 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:58.449261 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:58.451048 (+ 1787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:58.451785 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:58.451794 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:58.452313 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:01:58.453017 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:58.453037 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.454174 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.454180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:58.456195 (+ 2015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:58.456221 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.457240 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.457251 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:58.457531 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:58.458213 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:01:58.458236 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:01:58.460998 (+ 2762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:01:58.465734 (+ 4736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:58.465742 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:58.465755 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:58.466229 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:58.466734 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:01:58.466740 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:01:58.466745 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:01:58.467012 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:01:58.467707 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:01:58.467713 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:01:58.468075 (+ 362us) client_negotiation.cc:769] Sending connection context
0810 01:01:58.468372 (+ 297us) client_negotiation.cc:240] Negotiation successful
0810 01:01:58.468759 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":285,"threads_started":1}
I20260810 01:01:58.471500 7097 heartbeater.cc:347] Connected to a master server at 127.20.81.190:44961
I20260810 01:01:58.471834 7097 heartbeater.cc:464] Registering TS with master...
I20260810 01:01:58.472786 7097 heartbeater.cc:511] Master 127.20.81.190:44961 requested a full tablet report, sending...
I20260810 01:01:58.473138 7101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:58.455811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48298 (local address 127.20.81.190:44961)
0810 01:01:58.456783 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:58.456788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:58.456802 (+ 14us) server_negotiation.cc:407] Connection header received
0810 01:01:58.456845 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:58.456848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:58.456919 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:58.457036 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:58.458649 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.459547 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.461548 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.462268 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.462377 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:58.466368 (+ 3991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:58.466374 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:58.466383 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:58.466457 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:58.469651 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:58.470434 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:58.470442 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:58.470447 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:58.470594 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:58.471549 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:58.471554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:58.471556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:58.471925 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:58.472186 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:58.472465 (+ 279us) server_negotiation.cc:299] Negotiation successful
0810 01:01:58.472705 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":865,"thread_start_us":692,"threads_started":1}
I20260810 01:01:58.474457 6594 ts_manager.cc:194] Registered new tserver with Master: d5f9461b4f1d4474a34671fbc4a7d7f8 (127.20.81.131:36651)
I20260810 01:01:58.475298 6594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:42907
I20260810 01:01:58.475996 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 01:01:58.483654 7101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:58.476932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48306 (local address 127.20.81.190:44961)
0810 01:01:58.477116 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:58.477120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:58.478930 (+ 1810us) server_negotiation.cc:407] Connection header received
0810 01:01:58.479068 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:58.479071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:58.479199 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:58.479309 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:01:58.479744 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.480413 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.481882 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.482299 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.482392 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:58.482889 (+ 497us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:01:58.483129 (+ 240us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 01:01:58.483388 (+ 259us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48306: token expired
Metrics: {"server-negotiator.queue_time_us":50}
W20260810 01:01:58.483785 7101 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48306: token expired
W20260810 01:01:58.484371 6508 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.20.81.190:44961)
I20260810 01:01:58.497936 7101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:58.484862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48318 (local address 127.20.81.190:44961)
0810 01:01:58.485031 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:58.485037 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:58.485136 (+ 99us) server_negotiation.cc:407] Connection header received
0810 01:01:58.485347 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:58.485354 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:58.485426 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:58.485524 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:58.486969 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.487779 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.489237 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.489675 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.489806 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:58.492534 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:58.492541 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:58.492545 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:58.492584 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:58.495237 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:58.495941 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:58.495947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:58.495951 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:58.496082 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:58.496474 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:58.496479 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:58.496484 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:58.496781 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:58.497049 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:58.497349 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 01:01:58.497584 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 01:01:58.501252 6594 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 01:01:58.509207 6594 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 01:01:58.531417 6808 heartbeater.cc:503] Master 127.20.81.190:44961 was elected leader, sending a full tablet report...
I20260810 01:01:58.540777 7110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:58.521335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57628 (local address 127.20.81.131:36651)
0810 01:01:58.527338 (+ 6003us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:58.527349 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:58.527389 (+ 40us) server_negotiation.cc:407] Connection header received
0810 01:01:58.527507 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:58.527518 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:58.527808 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:58.530987 (+ 3179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:01:58.531343 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.533514 (+ 2171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.537560 (+ 4046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.539073 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.539199 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:58.539472 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:58.539941 (+ 469us) server_negotiation.cc:299] Negotiation successful
0810 01:01:58.540114 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5787,"thread_start_us":5635,"threads_started":1}
I20260810 01:01:58.541081 7108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:58.521043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:36651 (local address 127.0.0.1:57628)
0810 01:01:58.521938 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:58.521991 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:58.522189 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:58.528686 (+ 6497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:58.528698 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:58.528755 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:01:58.530667 (+ 1912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:58.530689 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.534333 (+ 3644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.534342 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:58.537399 (+ 3057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:58.537414 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.539305 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.539312 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:58.539556 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:58.539635 (+ 79us) client_negotiation.cc:769] Sending connection context
0810 01:01:58.539774 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 01:01:58.540632 (+ 858us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":23,"thread_start_us":889,"threads_started":1}
I20260810 01:01:58.543462 7109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:58.521139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:33271 (local address 127.0.0.1:33540)
0810 01:01:58.535600 (+ 14461us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:58.535627 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:58.535901 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:58.536672 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:58.536676 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:58.536701 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:01:58.537147 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:58.537164 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.538699 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.538706 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:58.540665 (+ 1959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:58.540683 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.542585 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.542594 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:58.542764 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:58.542796 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 01:01:58.542900 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 01:01:58.543037 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14283,"mutex_wait_us":4,"thread_start_us":14083,"threads_started":1}
I20260810 01:01:58.543462 7112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:58.521817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33540 (local address 127.20.81.129:33271)
0810 01:01:58.528500 (+ 6683us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:58.528512 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:58.535754 (+ 7242us) server_negotiation.cc:407] Connection header received
0810 01:01:58.536132 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:58.536138 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:58.536387 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:58.536565 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:01:58.537356 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.538564 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.540876 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.542360 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.542533 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:58.542768 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:58.542962 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 01:01:58.543126 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6482,"thread_start_us":6341,"threads_started":1}
I20260810 01:01:58.544787 7111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:58.521743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:36451 (local address 127.0.0.1:41304)
0810 01:01:58.536171 (+ 14428us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:58.536189 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:58.536306 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:58.536922 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:58.536927 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:58.536959 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:01:58.537554 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:58.537570 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.539483 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.539491 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:58.541730 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:58.541744 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.543966 (+ 2222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.543975 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:58.544144 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:58.544167 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 01:01:58.544271 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 01:01:58.544408 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14303,"mutex_wait_us":40,"thread_start_us":14121,"threads_started":1}
I20260810 01:01:58.545511 7026 tablet_service.cc:1467] Processing CreateTablet for tablet 4590a5dfad9344fea444af409509dbfa (DEFAULT_TABLE table=test-table [id=31d7e20e667e40dd8f6a9e5a0dc73e70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:01:58.547065 7026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4590a5dfad9344fea444af409509dbfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:01:58.547377 6733 tablet_service.cc:1467] Processing CreateTablet for tablet 4590a5dfad9344fea444af409509dbfa (DEFAULT_TABLE table=test-table [id=31d7e20e667e40dd8f6a9e5a0dc73e70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:01:58.548533 6733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4590a5dfad9344fea444af409509dbfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:01:58.550478 7114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:58.522028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41304 (local address 127.20.81.130:36451)
0810 01:01:58.534776 (+ 12748us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:58.534789 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:58.536308 (+ 1519us) server_negotiation.cc:407] Connection header received
0810 01:01:58.536452 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:58.536459 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:58.536707 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:58.536932 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:01:58.537736 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.539303 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.541921 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.543459 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.543603 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:58.543793 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:58.544315 (+ 522us) server_negotiation.cc:299] Negotiation successful
0810 01:01:58.544486 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12557,"thread_start_us":12781,"threads_started":1}
I20260810 01:01:58.554313 6880 tablet_service.cc:1467] Processing CreateTablet for tablet 4590a5dfad9344fea444af409509dbfa (DEFAULT_TABLE table=test-table [id=31d7e20e667e40dd8f6a9e5a0dc73e70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:01:58.555747 6880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4590a5dfad9344fea444af409509dbfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:01:58.559790 7115 tablet_bootstrap.cc:492] T 4590a5dfad9344fea444af409509dbfa P d5f9461b4f1d4474a34671fbc4a7d7f8: Bootstrap starting.
I20260810 01:01:58.563540 7115 tablet_bootstrap.cc:654] T 4590a5dfad9344fea444af409509dbfa P d5f9461b4f1d4474a34671fbc4a7d7f8: Neither blocks nor log segments found. Creating new log.
I20260810 01:01:58.564570 7115 log.cc:824] T 4590a5dfad9344fea444af409509dbfa P d5f9461b4f1d4474a34671fbc4a7d7f8: Log is configured to *not* fsync() on all Append() calls
I20260810 01:01:58.565514 7116 tablet_bootstrap.cc:492] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302: Bootstrap starting.
I20260810 01:01:58.569226 7115 tablet_bootstrap.cc:492] T 4590a5dfad9344fea444af409509dbfa P d5f9461b4f1d4474a34671fbc4a7d7f8: No bootstrap required, opened a new log
I20260810 01:01:58.569223 7116 tablet_bootstrap.cc:654] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302: Neither blocks nor log segments found. Creating new log.
I20260810 01:01:58.569562 7115 ts_tablet_manager.cc:1397] T 4590a5dfad9344fea444af409509dbfa P d5f9461b4f1d4474a34671fbc4a7d7f8: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260810 01:01:58.570540 7116 log.cc:824] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302: Log is configured to *not* fsync() on all Append() calls
I20260810 01:01:58.571045 7119 tablet_bootstrap.cc:492] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31: Bootstrap starting.
I20260810 01:01:58.574524 7119 tablet_bootstrap.cc:654] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31: Neither blocks nor log segments found. Creating new log.
I20260810 01:01:58.575285 7116 tablet_bootstrap.cc:492] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302: No bootstrap required, opened a new log
I20260810 01:01:58.575560 7116 ts_tablet_manager.cc:1397] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.001s
I20260810 01:01:58.575553 7119 log.cc:824] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31: Log is configured to *not* fsync() on all Append() calls
I20260810 01:01:58.578032 7119 tablet_bootstrap.cc:492] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31: No bootstrap required, opened a new log
I20260810 01:01:58.578289 7119 ts_tablet_manager.cc:1397] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260810 01:01:58.580839 7115 raft_consensus.cc:348] T 4590a5dfad9344fea444af409509dbfa P d5f9461b4f1d4474a34671fbc4a7d7f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } }
I20260810 01:01:58.581296 7115 raft_consensus.cc:374] T 4590a5dfad9344fea444af409509dbfa P d5f9461b4f1d4474a34671fbc4a7d7f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:01:58.581444 7115 raft_consensus.cc:729] T 4590a5dfad9344fea444af409509dbfa P d5f9461b4f1d4474a34671fbc4a7d7f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5f9461b4f1d4474a34671fbc4a7d7f8, State: Initialized, Role: FOLLOWER
I20260810 01:01:58.582084 7115 consensus_queue.cc:260] T 4590a5dfad9344fea444af409509dbfa P d5f9461b4f1d4474a34671fbc4a7d7f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } }
I20260810 01:01:58.583261 7097 heartbeater.cc:503] Master 127.20.81.190:44961 was elected leader, sending a full tablet report...
I20260810 01:01:58.583873 7115 ts_tablet_manager.cc:1428] T 4590a5dfad9344fea444af409509dbfa P d5f9461b4f1d4474a34671fbc4a7d7f8: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260810 01:01:58.586460 7116 raft_consensus.cc:348] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } }
I20260810 01:01:58.586941 7116 raft_consensus.cc:374] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:01:58.587055 7116 raft_consensus.cc:729] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eec9553789ba459c869a9fa7b56a7302, State: Initialized, Role: FOLLOWER
I20260810 01:01:58.586828 7119 raft_consensus.cc:348] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } }
I20260810 01:01:58.587306 7119 raft_consensus.cc:374] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:01:58.587467 7119 raft_consensus.cc:729] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24f57e6e4bc9455fa63a2770cf8c9a31, State: Initialized, Role: FOLLOWER
I20260810 01:01:58.588126 7116 consensus_queue.cc:260] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } }
I20260810 01:01:58.588239 7119 consensus_queue.cc:260] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } }
I20260810 01:01:58.590063 7116 ts_tablet_manager.cc:1428] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302: Time spent starting tablet: real 0.014s user 0.011s sys 0.003s
I20260810 01:01:58.591118 6955 heartbeater.cc:503] Master 127.20.81.190:44961 was elected leader, sending a full tablet report...
I20260810 01:01:58.591638 7119 ts_tablet_manager.cc:1428] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31: Time spent starting tablet: real 0.013s user 0.007s sys 0.006s
W20260810 01:01:58.697145 7098 tablet.cc:2369] T 4590a5dfad9344fea444af409509dbfa P d5f9461b4f1d4474a34671fbc4a7d7f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 01:01:58.713661 6956 tablet.cc:2369] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:01:58.732069 7125 raft_consensus.cc:482] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:01:58.732398 7125 raft_consensus.cc:504] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } }
I20260810 01:01:58.734187 7125 leader_election.cc:288] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5f9461b4f1d4474a34671fbc4a7d7f8 (127.20.81.131:36651), eec9553789ba459c869a9fa7b56a7302 (127.20.81.129:33271)
I20260810 01:01:58.743888 7112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:58.734857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:36003 (local address 127.20.81.129:33271)
0810 01:01:58.735498 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:58.735507 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:58.736063 (+ 556us) server_negotiation.cc:407] Connection header received
0810 01:01:58.736181 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:58.736186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:58.736262 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:58.736362 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:01:58.737171 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.738259 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.741349 (+ 3090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.742618 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.742788 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:58.742863 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:58.743501 (+ 638us) server_negotiation.cc:299] Negotiation successful
0810 01:01:58.743634 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445}
I20260810 01:01:58.743884 7110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:58.734779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:35587 (local address 127.20.81.131:36651)
0810 01:01:58.735255 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:58.735549 (+ 294us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:58.735713 (+ 164us) server_negotiation.cc:407] Connection header received
0810 01:01:58.735984 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:58.735988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:58.736078 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:58.736182 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:01:58.737011 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.738195 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.741349 (+ 3154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.742625 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.742788 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:58.742863 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:58.743508 (+ 645us) server_negotiation.cc:299] Negotiation successful
0810 01:01:58.743634 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260810 01:01:58.744100 7126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:58.734588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:36651 (local address 127.20.81.130:35587)
0810 01:01:58.735571 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:58.735599 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:58.735806 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:58.736233 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:58.736237 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:58.736257 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:01:58.736843 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:58.736862 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.738431 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.738434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:58.741168 (+ 2734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:58.741186 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.742922 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.742928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:58.743232 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:58.743329 (+ 97us) client_negotiation.cc:769] Sending connection context
0810 01:01:58.743473 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 01:01:58.743573 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"mutex_wait_us":215,"spinlock_wait_cycles":24320,"thread_start_us":611,"threads_started":1}
I20260810 01:01:58.744125 7127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:58.734678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:33271 (local address 127.20.81.130:36003)
0810 01:01:58.735958 (+ 1280us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:58.735982 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 01:01:58.736080 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:01:58.736398 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:01:58.736405 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:01:58.736448 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:01:58.737013 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:58.737028 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.738393 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.738421 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:58.741175 (+ 2754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:01:58.741186 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.742919 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.742925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:01:58.743235 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:58.743326 (+ 91us) client_negotiation.cc:769] Sending connection context
0810 01:01:58.743427 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 01:01:58.743566 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1128,"mutex_wait_us":258,"thread_start_us":894,"threads_started":1}
W20260810 01:01:58.744534 6809 tablet.cc:2369] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:01:58.745569 7047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4590a5dfad9344fea444af409509dbfa" candidate_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" is_pre_election: true
I20260810 01:01:58.745594 6756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4590a5dfad9344fea444af409509dbfa" candidate_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eec9553789ba459c869a9fa7b56a7302" is_pre_election: true
I20260810 01:01:58.746102 6756 raft_consensus.cc:2454] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24f57e6e4bc9455fa63a2770cf8c9a31 in term 0.
I20260810 01:01:58.746134 7047 raft_consensus.cc:2454] T 4590a5dfad9344fea444af409509dbfa P d5f9461b4f1d4474a34671fbc4a7d7f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24f57e6e4bc9455fa63a2770cf8c9a31 in term 0.
I20260810 01:01:58.746932 6833 leader_election.cc:302] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 24f57e6e4bc9455fa63a2770cf8c9a31, d5f9461b4f1d4474a34671fbc4a7d7f8, eec9553789ba459c869a9fa7b56a7302; no voters:
I20260810 01:01:58.747424 7125 raft_consensus.cc:2781] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:01:58.747606 7125 raft_consensus.cc:482] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:01:58.747716 7125 raft_consensus.cc:3037] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:01:58.749924 7125 raft_consensus.cc:504] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } }
I20260810 01:01:58.750676 7125 leader_election.cc:288] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31 [CANDIDATE]: Term 1 election: Requested vote from peers d5f9461b4f1d4474a34671fbc4a7d7f8 (127.20.81.131:36651), eec9553789ba459c869a9fa7b56a7302 (127.20.81.129:33271)
I20260810 01:01:58.750999 7047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4590a5dfad9344fea444af409509dbfa" candidate_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8"
I20260810 01:01:58.751206 7047 raft_consensus.cc:3037] T 4590a5dfad9344fea444af409509dbfa P d5f9461b4f1d4474a34671fbc4a7d7f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:01:58.751232 6756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4590a5dfad9344fea444af409509dbfa" candidate_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eec9553789ba459c869a9fa7b56a7302"
I20260810 01:01:58.751500 6756 raft_consensus.cc:3037] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:01:58.753780 7047 raft_consensus.cc:2454] T 4590a5dfad9344fea444af409509dbfa P d5f9461b4f1d4474a34671fbc4a7d7f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f57e6e4bc9455fa63a2770cf8c9a31 in term 1.
I20260810 01:01:58.754146 6833 leader_election.cc:302] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f57e6e4bc9455fa63a2770cf8c9a31, d5f9461b4f1d4474a34671fbc4a7d7f8; no voters:
I20260810 01:01:58.754455 7125 raft_consensus.cc:2781] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:01:58.754446 6756 raft_consensus.cc:2454] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f57e6e4bc9455fa63a2770cf8c9a31 in term 1.
I20260810 01:01:58.755035 7125 raft_consensus.cc:686] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31 [term 1 LEADER]: Becoming Leader. State: Replica: 24f57e6e4bc9455fa63a2770cf8c9a31, State: Running, Role: LEADER
I20260810 01:01:58.755635 7125 consensus_queue.cc:237] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } }
I20260810 01:01:58.760499 6593 catalog_manager.cc:5495] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24f57e6e4bc9455fa63a2770cf8c9a31 (127.20.81.130). New cstate: current_term: 1 leader_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } health_report { overall_health: HEALTHY } } }
I20260810 01:01:58.831069 7114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:58.825131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41310 (local address 127.20.81.130:36451)
0810 01:01:58.825468 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:58.825473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:58.825596 (+ 123us) server_negotiation.cc:407] Connection header received
0810 01:01:58.825758 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:58.825762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:58.825868 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:58.826003 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:01:58.826369 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.827265 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.828731 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.829141 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.829236 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:58.829792 (+ 556us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:01:58.830376 (+ 584us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:01:58.830571 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:58.830681 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 01:01:58.830752 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199}
I20260810 01:01:58.839917 6756 raft_consensus.cc:1261] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 1 FOLLOWER]: Refusing update from remote peer 24f57e6e4bc9455fa63a2770cf8c9a31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:01:58.839913 7047 raft_consensus.cc:1261] T 4590a5dfad9344fea444af409509dbfa P d5f9461b4f1d4474a34671fbc4a7d7f8 [term 1 FOLLOWER]: Refusing update from remote peer 24f57e6e4bc9455fa63a2770cf8c9a31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:01:58.841043 7128 consensus_queue.cc:1035] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:01:58.841405 7125 consensus_queue.cc:1035] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:01:58.868777 6594 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:01:58.869093 6594 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:01:58.872936 6594 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913: Sending DeleteTablet for 3 replicas of tablet 4590a5dfad9344fea444af409509dbfa
I20260810 01:01:58.874300 7026 tablet_service.cc:1514] Processing DeleteTablet for tablet 4590a5dfad9344fea444af409509dbfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:01:58 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:57628
I20260810 01:01:58.874300 6733 tablet_service.cc:1514] Processing DeleteTablet for tablet 4590a5dfad9344fea444af409509dbfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:01:58 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:33540
I20260810 01:01:58.874586 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 6636
I20260810 01:01:58.876000 6880 tablet_service.cc:1514] Processing DeleteTablet for tablet 4590a5dfad9344fea444af409509dbfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:01:58 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:41304
I20260810 01:01:58.887122 7142 tablet_replica.cc:330] T 4590a5dfad9344fea444af409509dbfa P d5f9461b4f1d4474a34671fbc4a7d7f8: stopping tablet replica
I20260810 01:01:58.887804 7142 raft_consensus.cc:2229] T 4590a5dfad9344fea444af409509dbfa P d5f9461b4f1d4474a34671fbc4a7d7f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:01:58.888592 7142 raft_consensus.cc:2258] T 4590a5dfad9344fea444af409509dbfa P d5f9461b4f1d4474a34671fbc4a7d7f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:01:58.889621 7143 tablet_replica.cc:330] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302: stopping tablet replica
I20260810 01:01:58.890236 7143 raft_consensus.cc:2229] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:01:58.892124 7142 ts_tablet_manager.cc:1905] T 4590a5dfad9344fea444af409509dbfa P d5f9461b4f1d4474a34671fbc4a7d7f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:01:58.894938 7143 raft_consensus.cc:2258] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:01:58.901226 7112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:58.875381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33546 (local address 127.20.81.129:33271)
0810 01:01:58.875578 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:58.875587 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:58.876179 (+ 592us) server_negotiation.cc:407] Connection header received
0810 01:01:58.876263 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:58.876272 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:58.876400 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:58.876523 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:58.879320 (+ 2797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.880989 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.885887 (+ 4898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:58.886876 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:58.886996 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:58.892199 (+ 5203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:58.892217 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:58.892238 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:58.892284 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:58.896565 (+ 4281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:58.897560 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:58.897569 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:58.897573 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:58.897758 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:58.898328 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:58.898334 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:58.898338 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:58.898851 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:58.899181 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:58.900509 (+ 1328us) server_negotiation.cc:299] Negotiation successful
0810 01:01:58.900744 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 01:01:58.901185 7142 ts_tablet_manager.cc:1918] T 4590a5dfad9344fea444af409509dbfa P d5f9461b4f1d4474a34671fbc4a7d7f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:01:58.901383 7142 log.cc:1196] T 4590a5dfad9344fea444af409509dbfa P d5f9461b4f1d4474a34671fbc4a7d7f8: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/4590a5dfad9344fea444af409509dbfa
I20260810 01:01:58.902395 6803 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:58.902889 7144 tablet_replica.cc:330] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31: stopping tablet replica
I20260810 01:01:58.903483 7144 raft_consensus.cc:2229] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31 [term 1 LEADER]: Raft consensus shutting down.
I20260810 01:01:58.904368 7144 raft_consensus.cc:2258] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:01:58.904340 7142 ts_tablet_manager.cc:1939] T 4590a5dfad9344fea444af409509dbfa P d5f9461b4f1d4474a34671fbc4a7d7f8: Deleting consensus metadata
I20260810 01:01:58.906584 6581 catalog_manager.cc:4841] TS d5f9461b4f1d4474a34671fbc4a7d7f8 (127.20.81.131:36651): tablet 4590a5dfad9344fea444af409509dbfa (table test-table [id=31d7e20e667e40dd8f6a9e5a0dc73e70]) successfully deleted
I20260810 01:01:58.907981 7144 ts_tablet_manager.cc:1905] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:01:58.916817 7144 ts_tablet_manager.cc:1918] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:01:58.917065 7144 log.cc:1196] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/4590a5dfad9344fea444af409509dbfa
I20260810 01:01:58.917723 7144 ts_tablet_manager.cc:1939] T 4590a5dfad9344fea444af409509dbfa P 24f57e6e4bc9455fa63a2770cf8c9a31: Deleting consensus metadata
I20260810 01:01:58.919946 6578 catalog_manager.cc:4841] TS 24f57e6e4bc9455fa63a2770cf8c9a31 (127.20.81.130:36451): tablet 4590a5dfad9344fea444af409509dbfa (table test-table [id=31d7e20e667e40dd8f6a9e5a0dc73e70]) successfully deleted
I20260810 01:01:58.903669 7143 ts_tablet_manager.cc:1905] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:01:59.018810 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 6636
I20260810 01:01:59.042804 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 6812
W20260810 01:01:59.042783 6579 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.81.129:33271 (error 108)
I20260810 01:01:59.045101 7111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:59.044582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:33271 (local address 127.0.0.1:33562)
0810 01:01:59.044784 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:59.044917 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:33271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260810 01:01:59.045400 6579 catalog_manager.cc:4584] TS eec9553789ba459c869a9fa7b56a7302 (127.20.81.129:33271): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4590a5dfad9344fea444af409509dbfa: Network error: Client connection negotiation failed: client connection to 127.20.81.129:33271: connect: Connection refused (error 111)
I20260810 01:01:59.060433 7114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:59.043663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41314 (local address 127.20.81.130:36451)
0810 01:01:59.043863 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:59.043869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:59.043995 (+ 126us) server_negotiation.cc:407] Connection header received
0810 01:01:59.044220 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:59.044227 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:59.044353 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:59.044474 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:59.046132 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:59.047407 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:59.048899 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:59.049616 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:59.049733 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:59.052617 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:59.052636 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:59.052657 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:59.052711 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:59.057140 (+ 4429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:59.057976 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:59.057988 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:59.057992 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:59.058182 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:59.058638 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:59.058644 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:59.058647 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:59.059133 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:59.059473 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:59.059772 (+ 299us) server_negotiation.cc:299] Negotiation successful
0810 01:01:59.060028 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 01:01:59.061311 6948 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:59.099066 7111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:59.098596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:33271 (local address 127.0.0.1:33578)
0810 01:01:59.098813 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:59.098915 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:33271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260810 01:01:59.100139 7111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:59.099770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:33271 (local address 127.0.0.1:33586)
0810 01:01:59.099932 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:59.100022 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:33271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 01:01:59.161028 7111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:59.160516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:33271 (local address 127.0.0.1:33594)
0810 01:01:59.160736 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:59.160837 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:33271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260810 01:01:59.162142 7111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:59.161739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:33271 (local address 127.0.0.1:33604)
0810 01:01:59.161910 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:59.161981 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:33271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 01:01:59.178104 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 6812
I20260810 01:01:59.203121 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 6958
I20260810 01:01:59.216789 7110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:59.203889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57644 (local address 127.20.81.131:36651)
0810 01:01:59.204068 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 01:01:59.204073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:01:59.204092 (+ 19us) server_negotiation.cc:407] Connection header received
0810 01:01:59.204191 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:01:59.204195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:01:59.204257 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:01:59.204342 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:01:59.205758 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:59.206531 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:59.207775 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:01:59.208157 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:01:59.208256 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:01:59.211134 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:01:59.211148 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:01:59.211163 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:01:59.211201 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:01:59.214079 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:59.214857 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:59.214863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:59.214866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:59.215030 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:01:59.215354 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:01:59.215357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:01:59.215359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:01:59.215727 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:01:59.215943 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 01:01:59.216232 (+ 289us) server_negotiation.cc:299] Negotiation successful
0810 01:01:59.216401 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 01:01:59.217461 7092 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:59.242942 7111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:59.242359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:33271 (local address 127.0.0.1:33618)
0810 01:01:59.242648 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:59.242749 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:33271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
I20260810 01:01:59.244210 7111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:59.243756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:33271 (local address 127.0.0.1:33626)
0810 01:01:59.244004 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:59.244085 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:33271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260810 01:01:59.313364 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 6958
I20260810 01:01:59.335678 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 6557
I20260810 01:01:59.337137 6629 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:01:59.457029 7111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:01:59.456408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:33271 (local address 127.0.0.1:33636)
0810 01:01:59.456686 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0810 01:01:59.456790 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:33271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260810 01:01:59.457269 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 6557
I20260810 01:02:19.483273 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:44961
--webserver_interface=127.20.81.190
--webserver_port=42755
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:38185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:44961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:02:19.835367 7150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:19.835724 7150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:19.835883 7150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:19.843098 7150 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 01:02:19.843233 7150 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 01:02:19.843326 7150 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 01:02:19.843425 7150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:02:19.843523 7150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:19.843623 7150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:19.843712 7150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:02:19.843792 7150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:02:19.854259 7150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:38185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:44961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:44961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=42755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:19.855805 7150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:19.857224 7150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:19.869707 7156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:19.869664 7158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:19.869750 7155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:19.870541 7150 server_base.cc:1034] running on GCE node
I20260810 01:02:19.871379 7150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:19.872825 7150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:19.874087 7150 hybrid_clock.cc:648] HybridClock initialized: now 1786323739874019 us; error 42 us; skew 500 ppm
I20260810 01:02:19.874498 7150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:02:19 dist-test-slave-gp6n krb5kdc[5940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323739, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:02:19.877851 7150 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 01:02:19.879381 7150 webserver.cc:460] Webserver started at http://127.20.81.190:42755/ using document root <none> and password file <none>
I20260810 01:02:19.880101 7150 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:19.880234 7150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:19.885826 7150 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 01:02:19.888716 7168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:19.890142 7150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 01:02:19.890352 7150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "15a422ff679047df91331709f6a13913"
format_stamp: "Formatted at 2026-08-10 01:01:33 on dist-test-slave-gp6n"
server_key: "8517a733b5ee905f00fbbb9b83c4158c"
server_key_iv: "5040858cf93f24c30655b13fab6ebb3a"
server_key_version: "encryptionkey@0"
I20260810 01:02:19.891398 7150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:19.903563 7150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:19.907905 7150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:19.908207 7150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:19.927623 7150 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:44961
I20260810 01:02:19.927675 7226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:44961 every 8 connection(s)
I20260810 01:02:19.929287 7150 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:02:19.936784 7227 sys_catalog.cc:263] Verifying existing consensus state
I20260810 01:02:19.937013 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 7150
I20260810 01:02:19.938093 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:33271
--local_ip_for_outbound_sockets=127.20.81.129
--tserver_master_addrs=127.20.81.190:44961
--webserver_port=45695
--webserver_interface=127.20.81.129
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:38185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 01:02:19.942106 7227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913: Bootstrap starting.
I20260810 01:02:19.966861 7227 log.cc:824] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:19.983762 7227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 01:02:19.984364 7227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913: Bootstrap complete.
I20260810 01:02:19.993678 7227 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a422ff679047df91331709f6a13913" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44961 } }
I20260810 01:02:19.994656 7227 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15a422ff679047df91331709f6a13913, State: Initialized, Role: FOLLOWER
I20260810 01:02:19.995450 7227 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a422ff679047df91331709f6a13913" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44961 } }
I20260810 01:02:19.995723 7227 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:02:19.995857 7227 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:02:19.996060 7227 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 2 FOLLOWER]: Advancing to term 3
I20260810 01:02:20.000154 7227 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a422ff679047df91331709f6a13913" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44961 } }
I20260810 01:02:20.000674 7227 leader_election.cc:302] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15a422ff679047df91331709f6a13913; no voters:
I20260810 01:02:20.001390 7227 leader_election.cc:288] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 01:02:20.001550 7231 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 01:02:20.002753 7231 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [term 3 LEADER]: Becoming Leader. State: Replica: 15a422ff679047df91331709f6a13913, State: Running, Role: LEADER
I20260810 01:02:20.003394 7231 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a422ff679047df91331709f6a13913" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44961 } }
I20260810 01:02:20.003809 7227 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:02:20.006119 7232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "15a422ff679047df91331709f6a13913" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a422ff679047df91331709f6a13913" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44961 } } }
I20260810 01:02:20.006088 7236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15a422ff679047df91331709f6a13913. Latest consensus state: current_term: 3 leader_uuid: "15a422ff679047df91331709f6a13913" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a422ff679047df91331709f6a13913" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44961 } } }
I20260810 01:02:20.006656 7232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [sys.catalog]: This master's current role is: LEADER
I20260810 01:02:20.006714 7236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913 [sys.catalog]: This master's current role is: LEADER
I20260810 01:02:20.010890 7240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:02:20.022533 7240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:02:20.024507 7240 catalog_manager.cc:1259] Loaded cluster ID: 7ce9c0c0e27d4831bef4f47cee0b273c
I20260810 01:02:20.024690 7240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:02:20.030211 7240 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:02:20.032366 7240 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913: Loaded TSK: 0
I20260810 01:02:20.032595 7240 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 15a422ff679047df91331709f6a13913: Loaded TSK: 1
F20260810 01:02:20.034054 7240 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f772d6ab9ad google::LogMessage::Fail() at ??:0
@ 0x7f772d6ae88a google::LogMessage::SendToLog() at ??:0
@ 0x7f772d6ab4de google::LogMessage::Flush() at ??:0
@ 0x7f772d6aca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f773140f7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f77313fe73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f77430da1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f77430d9783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f77431652a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f77431650c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f77431af5e9 std::function<>::operator()() at ??:0
@ 0x7f77430ddcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f77430dbc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f7743163610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f7743163439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f77302f7345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f77302fa91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f77302fa799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f77302d5eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f77335af6db start_thread at ??:0
@ 0x7f772c54561f clone at ??:0
W20260810 01:02:20.358479 7229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:20.358764 7229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:20.358930 7229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:20.366302 7229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:20.366482 7229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:20.366709 7229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 01:02:20.376775 7229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:38185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:33271
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=45695
--tserver_master_addrs=127.20.81.190:44961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:20.378145 7229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:20.379539 7229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:20.393078 7281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:20.393203 7282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:20.393167 7284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:20.394484 7229 server_base.cc:1034] running on GCE node
I20260810 01:02:20.395139 7229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:20.396102 7229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:20.397348 7229 hybrid_clock.cc:648] HybridClock initialized: now 1786323740397296 us; error 45 us; skew 500 ppm
I20260810 01:02:20.397702 7229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:02:20 dist-test-slave-gp6n krb5kdc[5940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323740, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:02:20.405633 7229 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 01:02:20.407373 7229 webserver.cc:460] Webserver started at http://127.20.81.129:45695/ using document root <none> and password file <none>
I20260810 01:02:20.408192 7229 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:20.408350 7229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:20.414011 7229 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260810 01:02:20.417052 7291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:20.418699 7229 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 01:02:20.418922 7229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "eec9553789ba459c869a9fa7b56a7302"
format_stamp: "Formatted at 2026-08-10 01:01:34 on dist-test-slave-gp6n"
server_key: "97459b84813e059360046487bc9538af"
server_key_iv: "c5fdf0988dc90e94e7d9caba44a13dec"
server_key_version: "encryptionkey@0"
I20260810 01:02:20.420114 7229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:20.436446 7229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:20.441020 7229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:20.441349 7229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:20.442543 7229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:02:20.444615 7298 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260810 01:02:20.450659 7229 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 01:02:20.450822 7229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.001s
I20260810 01:02:20.450985 7229 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 01:02:20.454358 7229 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 01:02:20.454504 7229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.003s sys 0.001s
I20260810 01:02:20.454844 7298 tablet_bootstrap.cc:492] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302: Bootstrap starting.
I20260810 01:02:20.463263 7298 log.cc:824] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:20.480511 7298 tablet_bootstrap.cc:492] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 01:02:20.482678 7298 tablet_bootstrap.cc:492] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302: Bootstrap complete.
I20260810 01:02:20.483407 7298 ts_tablet_manager.cc:1397] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302: Time spent bootstrapping tablet: real 0.029s user 0.018s sys 0.005s
I20260810 01:02:20.496632 7298 raft_consensus.cc:348] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } }
I20260810 01:02:20.498497 7298 raft_consensus.cc:729] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eec9553789ba459c869a9fa7b56a7302, State: Initialized, Role: FOLLOWER
I20260810 01:02:20.499661 7298 consensus_queue.cc:260] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } }
I20260810 01:02:20.501819 7298 ts_tablet_manager.cc:1428] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302: Time spent starting tablet: real 0.018s user 0.013s sys 0.004s
I20260810 01:02:20.509387 7229 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:33271
I20260810 01:02:20.509435 7423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:33271 every 8 connection(s)
I20260810 01:02:20.510697 7229 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 01:02:20.515070 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 7229
I20260810 01:02:20.515728 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:36451
--local_ip_for_outbound_sockets=127.20.81.130
--tserver_master_addrs=127.20.81.190:44961
--webserver_port=37045
--webserver_interface=127.20.81.130
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:38185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 01:02:20.515780 7427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:20.513761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.129:57561)
0810 01:02:20.514598 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:20.514848 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":315,"threads_started":1}
W20260810 01:02:20.516490 7424 heartbeater.cc:644] Failed to heartbeat to 127.20.81.190:44961 (0 consecutive failures): Network error: Failed to ping master at 127.20.81.190:44961: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
I20260810 01:02:20.517490 7427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:20.517028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.129:44103)
0810 01:02:20.517224 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:20.517323 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 01:02:20.518641 7427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:20.518158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.129:44711)
0810 01:02:20.518340 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:20.518474 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 01:02:20.518918 7424 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 01:02:20.872545 7428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:20.872900 7428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:20.873075 7428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:20.880491 7428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:20.880651 7428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:20.880834 7428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 01:02:20.890854 7428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:38185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:36451
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=37045
--tserver_master_addrs=127.20.81.190:44961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:20.892256 7428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:20.893708 7428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:20.907507 7438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:20.907929 7435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:20.907720 7436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:20.908712 7428 server_base.cc:1034] running on GCE node
I20260810 01:02:20.909353 7428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:20.910323 7428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:20.911554 7428 hybrid_clock.cc:648] HybridClock initialized: now 1786323740911511 us; error 41 us; skew 500 ppm
I20260810 01:02:20.911887 7428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:02:20 dist-test-slave-gp6n krb5kdc[5940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323740, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:02:20.915206 7428 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 01:02:20.916821 7428 webserver.cc:460] Webserver started at http://127.20.81.130:37045/ using document root <none> and password file <none>
I20260810 01:02:20.917518 7428 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:20.917670 7428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:20.923285 7428 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.002s
I20260810 01:02:20.926292 7446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:20.927745 7428 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 01:02:20.927937 7428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "24f57e6e4bc9455fa63a2770cf8c9a31"
format_stamp: "Formatted at 2026-08-10 01:01:34 on dist-test-slave-gp6n"
server_key: "f646330df02b2bce4cb8b12179c3191f"
server_key_iv: "d836eaafbc882efbf0d8f72f78e4efa1"
server_key_version: "encryptionkey@0"
I20260810 01:02:20.928967 7428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:20.960186 7428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:20.964344 7428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:20.964639 7428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:20.965696 7428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:02:20.967136 7428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:02:20.967260 7428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:20.967393 7428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:02:20.967494 7428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:21.002717 7428 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:36451
I20260810 01:02:21.002751 7571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:36451 every 8 connection(s)
I20260810 01:02:21.004241 7428 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 01:02:21.009068 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 7428
I20260810 01:02:21.009490 7575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:21.007485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.130:43023)
0810 01:02:21.008307 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:21.008560 (+ 253us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":323,"threads_started":1}
I20260810 01:02:21.009788 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:36651
--local_ip_for_outbound_sockets=127.20.81.131
--tserver_master_addrs=127.20.81.190:44961
--webserver_port=33563
--webserver_interface=127.20.81.131
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:38185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:02:21.010501 7572 heartbeater.cc:644] Failed to heartbeat to 127.20.81.190:44961 (0 consecutive failures): Network error: Failed to ping master at 127.20.81.190:44961: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
I20260810 01:02:21.011480 7575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:21.011095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.130:56023)
0810 01:02:21.011280 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:21.011371 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260810 01:02:21.012418 7575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:21.012040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.130:33601)
0810 01:02:21.012182 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:21.012270 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 01:02:21.012681 7572 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 01:02:21.360162 7576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:21.360484 7576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:21.360589 7576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:21.367839 7576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:21.367959 7576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:21.368074 7576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 01:02:21.377805 7576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:38185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:36651
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=33563
--tserver_master_addrs=127.20.81.190:44961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:21.379097 7576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:21.380394 7576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:21.394325 7585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:21.394304 7581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:21.394491 7582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:21.395601 7576 server_base.cc:1034] running on GCE node
I20260810 01:02:21.396325 7576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:21.397439 7576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:21.398661 7576 hybrid_clock.cc:648] HybridClock initialized: now 1786323741398622 us; error 41 us; skew 500 ppm
I20260810 01:02:21.399019 7576 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:02:21 dist-test-slave-gp6n krb5kdc[5940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323741, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:02:21.402334 7576 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 01:02:21.403873 7576 webserver.cc:460] Webserver started at http://127.20.81.131:33563/ using document root <none> and password file <none>
I20260810 01:02:21.404595 7576 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:21.404750 7576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:21.410517 7576 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.003s
I20260810 01:02:21.413489 7593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:21.414878 7576 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 01:02:21.415079 7576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8"
format_stamp: "Formatted at 2026-08-10 01:01:35 on dist-test-slave-gp6n"
server_key: "526433a6d191504e14fda4d038284f63"
server_key_iv: "07c73ada98560d66a4ea79d24bcc8857"
server_key_version: "encryptionkey@0"
I20260810 01:02:21.416090 7576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:21.442456 7576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:21.446698 7576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:21.446995 7576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:21.448040 7576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:02:21.449450 7576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:02:21.449589 7576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:21.449713 7576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:02:21.449802 7576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:21.484972 7576 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:36651
I20260810 01:02:21.485060 7721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:36651 every 8 connection(s)
I20260810 01:02:21.486517 7576 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 01:02:21.491351 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:21.489462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.131:45629)
0810 01:02:21.490183 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:21.490500 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":280,"threads_started":1}
W20260810 01:02:21.492069 7722 heartbeater.cc:644] Failed to heartbeat to 127.20.81.190:44961 (0 consecutive failures): Network error: Failed to ping master at 127.20.81.190:44961: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
I20260810 01:02:21.492831 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 7576
I20260810 01:02:21.492882 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:21.492561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.131:57193)
0810 01:02:21.492719 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:21.492782 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 01:02:21.493780 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:21.493458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.131:34769)
0810 01:02:21.493600 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:21.493675 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260810 01:02:21.494122 7722 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 01:02:21.520288 7728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:21.519387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.129:33939)
0810 01:02:21.519897 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:21.519996 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":283,"threads_started":1}
I20260810 01:02:22.013989 7730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:22.013197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.130:46339)
0810 01:02:22.013663 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:22.013757 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":264,"threads_started":1}
I20260810 01:02:22.195348 7732 raft_consensus.cc:482] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:02:22.195708 7732 raft_consensus.cc:504] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } }
I20260810 01:02:22.197366 7732 leader_election.cc:288] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d5f9461b4f1d4474a34671fbc4a7d7f8 (127.20.81.131:36651), 24f57e6e4bc9455fa63a2770cf8c9a31 (127.20.81.130:36451)
Aug 10 01:02:22 dist-test-slave-gp6n krb5kdc[5940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323740, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.130@KRBTEST.COM
Aug 10 01:02:22 dist-test-slave-gp6n krb5kdc[5940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323740, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.131@KRBTEST.COM
I20260810 01:02:22.242522 7733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:22.197536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:36651 (local address 127.20.81.129:43631)
0810 01:02:22.212513 (+ 14977us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:22.212579 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:22.213359 (+ 780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:22.218532 (+ 5173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:22.218544 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:22.219452 (+ 908us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:02:22.220478 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:22.220510 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:22.223192 (+ 2682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:22.223198 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:22.226216 (+ 3018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:22.226229 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:22.227968 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:22.227975 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:22.228269 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:22.229213 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:02:22.229270 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:02:22.233586 (+ 4316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:02:22.238091 (+ 4505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:02:22.238101 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:02:22.238120 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:02:22.239061 (+ 941us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:02:22.239550 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:02:22.239557 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:02:22.239565 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:02:22.239758 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:02:22.240818 (+ 1060us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:02:22.240826 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:02:22.241179 (+ 353us) client_negotiation.cc:769] Sending connection context
0810 01:02:22.241465 (+ 286us) client_negotiation.cc:240] Negotiation successful
0810 01:02:22.242009 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14789,"mutex_wait_us":2,"thread_start_us":16210,"threads_started":1}
I20260810 01:02:22.242611 7735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:22.198185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:43631 (local address 127.20.81.131:36651)
0810 01:02:22.214297 (+ 16112us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:22.214315 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:22.214359 (+ 44us) server_negotiation.cc:407] Connection header received
0810 01:02:22.216734 (+ 2375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:22.216746 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:22.217276 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:22.217672 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:22.220795 (+ 3123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:22.222980 (+ 2185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:22.226398 (+ 3418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:22.227472 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:22.227663 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:22.233923 (+ 6260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:22.233935 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:22.233961 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:22.234001 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:22.237507 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:22.239217 (+ 1710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:22.239226 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:22.239235 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:22.239417 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:22.239891 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:22.239896 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:22.239900 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:22.240665 (+ 765us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:22.240877 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:22.241717 (+ 840us) server_negotiation.cc:299] Negotiation successful
0810 01:02:22.242153 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15891,"thread_start_us":16241,"threads_started":1}
I20260810 01:02:22.243535 7734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:22.197902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:36451 (local address 127.20.81.129:43713)
0810 01:02:22.214390 (+ 16488us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:22.214431 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:22.214652 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:22.220414 (+ 5762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:22.220420 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:22.221134 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:02:22.221538 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:22.221551 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:22.223877 (+ 2326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:22.223883 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:22.225790 (+ 1907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:22.225809 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:22.228703 (+ 2894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:22.228708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:22.228831 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:22.229557 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:02:22.229577 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:02:22.233101 (+ 3524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:02:22.239186 (+ 6085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:02:22.239195 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:02:22.239201 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:02:22.239910 (+ 709us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:02:22.240427 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:02:22.240436 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:02:22.240442 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:02:22.240671 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:02:22.242025 (+ 1354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:02:22.242033 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:02:22.242262 (+ 229us) client_negotiation.cc:769] Sending connection context
0810 01:02:22.242539 (+ 277us) client_negotiation.cc:240] Negotiation successful
0810 01:02:22.242922 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16352,"thread_start_us":16215,"threads_started":1}
I20260810 01:02:22.245226 7736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:22.198768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:43713 (local address 127.20.81.130:36451)
0810 01:02:22.207056 (+ 8288us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:22.207083 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:22.214537 (+ 7454us) server_negotiation.cc:407] Connection header received
0810 01:02:22.218487 (+ 3950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:22.218526 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:22.219158 (+ 632us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:22.219667 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:22.221742 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:22.223660 (+ 1918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:22.225996 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:22.228226 (+ 2230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:22.228366 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:22.233508 (+ 5142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:22.233526 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:22.233554 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:22.233630 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:22.238968 (+ 5338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:22.240074 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:22.240083 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:22.240091 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:22.240283 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:22.240836 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:22.240845 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:22.240851 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:22.241705 (+ 854us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:22.241919 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:22.244074 (+ 2155us) server_negotiation.cc:299] Negotiation successful
0810 01:02:22.244655 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8033,"thread_start_us":7814,"threads_started":1}
W20260810 01:02:22.246670 7295 leader_election.cc:341] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d5f9461b4f1d4474a34671fbc4a7d7f8 (127.20.81.131:36651): Not found: Tablet not found: 4590a5dfad9344fea444af409509dbfa
W20260810 01:02:22.248658 7292 leader_election.cc:341] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 24f57e6e4bc9455fa63a2770cf8c9a31 (127.20.81.130:36451): Not found: Tablet not found: 4590a5dfad9344fea444af409509dbfa
I20260810 01:02:22.248854 7292 leader_election.cc:302] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eec9553789ba459c869a9fa7b56a7302; no voters: 24f57e6e4bc9455fa63a2770cf8c9a31, d5f9461b4f1d4474a34671fbc4a7d7f8
I20260810 01:02:22.249282 7732 raft_consensus.cc:2726] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:02:22.495863 7737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:22.494717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.131:48801)
0810 01:02:22.495334 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:22.495556 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":281,"threads_started":1}
I20260810 01:02:22.521742 7734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:22.521237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.129:38829)
0810 01:02:22.521474 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:22.521568 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260810 01:02:23.015684 7738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:23.014977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.130:42535)
0810 01:02:23.015416 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:23.015519 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":230,"threads_started":1}
I20260810 01:02:23.497608 7739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:23.496827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.131:56879)
0810 01:02:23.497348 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:23.497442 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":249,"threads_started":1}
I20260810 01:02:23.523388 7740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:23.522608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.129:36529)
0810 01:02:23.523089 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:23.523172 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":252,"threads_started":1}
I20260810 01:02:24.017314 7741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:24.016565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.130:58681)
0810 01:02:24.017044 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:24.017133 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":272,"threads_started":1}
I20260810 01:02:24.249689 7742 raft_consensus.cc:482] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:02:24.249915 7742 raft_consensus.cc:504] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } }
I20260810 01:02:24.250597 7742 leader_election.cc:288] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d5f9461b4f1d4474a34671fbc4a7d7f8 (127.20.81.131:36651), 24f57e6e4bc9455fa63a2770cf8c9a31 (127.20.81.130:36451)
W20260810 01:02:24.251729 7295 leader_election.cc:341] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d5f9461b4f1d4474a34671fbc4a7d7f8 (127.20.81.131:36651): Not found: Tablet not found: 4590a5dfad9344fea444af409509dbfa
W20260810 01:02:24.251940 7292 leader_election.cc:341] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 24f57e6e4bc9455fa63a2770cf8c9a31 (127.20.81.130:36451): Not found: Tablet not found: 4590a5dfad9344fea444af409509dbfa
I20260810 01:02:24.252111 7292 leader_election.cc:302] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eec9553789ba459c869a9fa7b56a7302; no voters: 24f57e6e4bc9455fa63a2770cf8c9a31, d5f9461b4f1d4474a34671fbc4a7d7f8
I20260810 01:02:24.252393 7742 raft_consensus.cc:2726] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:02:24.499362 7743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:24.498638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.131:45373)
0810 01:02:24.499120 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:24.499202 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":248,"threads_started":1}
I20260810 01:02:24.525270 7744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:24.524408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.129:34979)
0810 01:02:24.524892 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:24.525003 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":234,"threads_started":1}
I20260810 01:02:25.019412 7745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:25.018499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.130:57707)
0810 01:02:25.019098 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:25.019220 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":305,"threads_started":1}
I20260810 01:02:25.501148 7746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:25.500410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.131:50315)
0810 01:02:25.500875 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:25.500964 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":234,"threads_started":1}
I20260810 01:02:25.527201 7747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:25.526340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.129:49875)
0810 01:02:25.526907 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:25.526996 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":226,"threads_started":1}
I20260810 01:02:26.021250 7748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:26.020519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.130:51267)
0810 01:02:26.021001 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:26.021086 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":247,"threads_started":1}
I20260810 01:02:26.502871 7749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:26.502076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.131:47665)
0810 01:02:26.502614 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:26.502699 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":271,"threads_started":1}
I20260810 01:02:26.528841 7750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:26.528146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.129:43201)
0810 01:02:26.528584 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:26.528669 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":230,"threads_started":1}
I20260810 01:02:27.023118 7751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:27.022255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.130:45863)
0810 01:02:27.022846 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:27.022936 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":337,"threads_started":1}
I20260810 01:02:27.426978 7752 raft_consensus.cc:482] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:02:27.427218 7752 raft_consensus.cc:504] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } }
I20260810 01:02:27.427917 7752 leader_election.cc:288] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d5f9461b4f1d4474a34671fbc4a7d7f8 (127.20.81.131:36651), 24f57e6e4bc9455fa63a2770cf8c9a31 (127.20.81.130:36451)
W20260810 01:02:27.428998 7292 leader_election.cc:341] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 24f57e6e4bc9455fa63a2770cf8c9a31 (127.20.81.130:36451): Not found: Tablet not found: 4590a5dfad9344fea444af409509dbfa
W20260810 01:02:27.429201 7295 leader_election.cc:341] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d5f9461b4f1d4474a34671fbc4a7d7f8 (127.20.81.131:36651): Not found: Tablet not found: 4590a5dfad9344fea444af409509dbfa
I20260810 01:02:27.429327 7295 leader_election.cc:302] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eec9553789ba459c869a9fa7b56a7302; no voters: 24f57e6e4bc9455fa63a2770cf8c9a31, d5f9461b4f1d4474a34671fbc4a7d7f8
I20260810 01:02:27.429566 7752 raft_consensus.cc:2726] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:02:27.504683 7753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:27.503939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.131:54955)
0810 01:02:27.504426 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:27.504506 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":248,"threads_started":1}
I20260810 01:02:27.530720 7754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:27.529781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.129:40879)
0810 01:02:27.530285 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:27.530387 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":271,"threads_started":1}
I20260810 01:02:28.024999 7755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:28.024086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.130:39185)
0810 01:02:28.024692 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:28.024790 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":258,"threads_started":1}
I20260810 01:02:28.506583 7756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:28.505638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.131:40327)
0810 01:02:28.506145 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:28.506226 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":220,"threads_started":1}
I20260810 01:02:28.532250 7757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:28.531565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.129:32857)
0810 01:02:28.531996 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:28.532096 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":211,"threads_started":1}
I20260810 01:02:29.026772 7758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:29.026003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.130:45255)
0810 01:02:29.026494 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:29.026588 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":221,"threads_started":1}
I20260810 01:02:29.508368 7759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:29.507536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.131:57527)
0810 01:02:29.508080 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:29.508163 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":335,"threads_started":1}
I20260810 01:02:29.534265 7760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:29.533263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.129:33465)
0810 01:02:29.533941 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:29.534071 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":303,"threads_started":1}
I20260810 01:02:30.028479 7761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:30.027650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.130:54819)
0810 01:02:30.028190 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:30.028286 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":259,"threads_started":1}
I20260810 01:02:30.510459 7762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:30.509487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.131:33975)
0810 01:02:30.510131 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:30.510253 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":322,"threads_started":1}
I20260810 01:02:30.536167 7763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:30.535399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.129:35695)
0810 01:02:30.535894 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:30.536000 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":259,"threads_started":1}
I20260810 01:02:31.030360 7764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:31.029539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.130:44827)
0810 01:02:31.030074 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:31.030172 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":281,"threads_started":1}
I20260810 01:02:31.325313 7765 raft_consensus.cc:482] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:02:31.325532 7765 raft_consensus.cc:504] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f9461b4f1d4474a34671fbc4a7d7f8" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 36651 } } peers { permanent_uuid: "eec9553789ba459c869a9fa7b56a7302" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 33271 } } peers { permanent_uuid: "24f57e6e4bc9455fa63a2770cf8c9a31" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 36451 } }
I20260810 01:02:31.326164 7765 leader_election.cc:288] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d5f9461b4f1d4474a34671fbc4a7d7f8 (127.20.81.131:36651), 24f57e6e4bc9455fa63a2770cf8c9a31 (127.20.81.130:36451)
W20260810 01:02:31.327399 7295 leader_election.cc:341] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d5f9461b4f1d4474a34671fbc4a7d7f8 (127.20.81.131:36651): Not found: Tablet not found: 4590a5dfad9344fea444af409509dbfa
W20260810 01:02:31.327612 7292 leader_election.cc:341] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 24f57e6e4bc9455fa63a2770cf8c9a31 (127.20.81.130:36451): Not found: Tablet not found: 4590a5dfad9344fea444af409509dbfa
I20260810 01:02:31.327811 7292 leader_election.cc:302] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eec9553789ba459c869a9fa7b56a7302; no voters: 24f57e6e4bc9455fa63a2770cf8c9a31, d5f9461b4f1d4474a34671fbc4a7d7f8
I20260810 01:02:31.328142 7765 raft_consensus.cc:2726] T 4590a5dfad9344fea444af409509dbfa P eec9553789ba459c869a9fa7b56a7302 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:02:31.513551 7766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:31.512457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.131:56435)
0810 01:02:31.513140 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:31.513276 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":331,"threads_started":1}
I20260810 01:02:31.537955 7767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:31.537110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.129:39705)
0810 01:02:31.537643 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:31.537759 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":268,"threads_started":1}
I20260810 01:02:32.032322 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:32.031399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.130:38523)
0810 01:02:32.031947 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:32.032051 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":271,"threads_started":1}
I20260810 01:02:32.515388 7769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:32.514637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.131:44337)
0810 01:02:32.515127 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:32.515225 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":233,"threads_started":1}
I20260810 01:02:32.539747 7770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:32.538959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.129:56253)
0810 01:02:32.539473 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:32.539564 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":277,"threads_started":1}
I20260810 01:02:33.034309 7771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:33.033295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.130:50355)
0810 01:02:33.034004 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:33.034113 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":365,"threads_started":1}
I20260810 01:02:33.517407 7773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:33.516478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.131:35663)
0810 01:02:33.517097 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:33.517202 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":412,"threads_started":1}
I20260810 01:02:33.541724 7774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:33.540733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.129:43077)
0810 01:02:33.541336 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:33.541445 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":346,"threads_started":1}
I20260810 01:02:34.036088 7775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:34.035304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.130:37557)
0810 01:02:34.035794 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:34.035902 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":262,"threads_started":1}
I20260810 01:02:34.519043 7776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:34.518296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.131:59637)
0810 01:02:34.518785 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:34.518876 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":274,"threads_started":1}
I20260810 01:02:34.543625 7777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:34.542825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.129:46449)
0810 01:02:34.543333 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:34.543426 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":251,"threads_started":1}
I20260810 01:02:35.037689 7778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:35.036966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.130:42053)
0810 01:02:35.037408 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:35.037489 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":230,"threads_started":1}
I20260810 01:02:35.520818 7779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:35.519964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.131:47365)
0810 01:02:35.520515 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:35.520600 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":308,"threads_started":1}
I20260810 01:02:35.545571 7781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:35.544799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.129:37485)
0810 01:02:35.545267 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:35.545383 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":225,"threads_started":1}
I20260810 01:02:36.039453 7783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:36.038755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.130:44261)
0810 01:02:36.039197 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:36.039275 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":213,"threads_started":1}
I20260810 01:02:36.505069 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 7229
I20260810 01:02:36.522845 7785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:36.521912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.131:51135)
0810 01:02:36.522493 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:36.522634 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":256,"threads_started":1}
I20260810 01:02:36.523501 7784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:36.506212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37686 (local address 127.20.81.129:33271)
0810 01:02:36.506856 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:36.506871 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:36.506909 (+ 38us) server_negotiation.cc:407] Connection header received
0810 01:02:36.506999 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:36.507012 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:36.507362 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:36.507587 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:36.509552 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:36.510725 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:36.512709 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:36.513364 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:36.513477 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:36.516808 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:36.516821 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:36.516835 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:36.516868 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:36.520117 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:36.520866 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:36.520876 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:36.520879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:36.521037 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:36.521433 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:36.521439 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:36.521443 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:36.521987 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:36.522320 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:36.522793 (+ 473us) server_negotiation.cc:299] Negotiation successful
0810 01:02:36.523106 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":294,"threads_started":1}
I20260810 01:02:36.524940 7419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:36.616916 7787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:36.615848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44961 (local address 127.20.81.129:53285)
0810 01:02:36.616448 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:36.616642 (+ 194us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.190:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":294,"threads_started":1}
I20260810 01:02:36.617113 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 7229
I20260810 01:02:36.642609 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 7428
I20260810 01:02:36.657302 7789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:36.643498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58498 (local address 127.20.81.130:36451)
0810 01:02:36.644079 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:36.644084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:36.644100 (+ 16us) server_negotiation.cc:407] Connection header received
0810 01:02:36.644156 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:36.644160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:36.644219 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:36.644345 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:36.645808 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:36.646695 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:36.647977 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:36.648439 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:36.648572 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:36.651592 (+ 3020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:36.651598 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:36.651602 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:36.651636 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:36.654219 (+ 2583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:36.655196 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:36.655204 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:36.655209 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:36.655368 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:36.655788 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:36.655793 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:36.655797 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:36.656083 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:36.656368 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:36.656663 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 01:02:36.656897 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":319,"threads_started":1}
I20260810 01:02:36.658063 7567 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:36.744478 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 7428
I20260810 01:02:36.766287 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 7576
I20260810 01:02:36.781404 7791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:36.767351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43774 (local address 127.20.81.131:36651)
0810 01:02:36.768016 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:36.768022 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:36.768039 (+ 17us) server_negotiation.cc:407] Connection header received
0810 01:02:36.768097 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:36.768102 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:36.768164 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:36.768258 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:36.769843 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:36.770882 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:36.772126 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:36.772647 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:36.772750 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:36.775610 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:36.775616 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:36.775620 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:36.775655 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:36.778786 (+ 3131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:36.779556 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:36.779561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:36.779564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:36.779688 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:36.780038 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:36.780041 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:36.780043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:36.780321 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:36.780549 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:36.780825 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 01:02:36.781007 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":382,"threads_started":1}
I20260810 01:02:36.782183 7717 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:36.867542 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 7576
2026-08-10T01:02:36Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65217 ms)
[----------] 30 tests from SecurityITest (311134 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:02:37 dist-test-slave-gp6n krb5kdc[7796](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:02:37 dist-test-slave-gp6n krb5kdc[7796](info): set up 2 sockets
Aug 10 01:02:37 dist-test-slave-gp6n krb5kdc[7796](info): commencing operation
krb5kdc: starting...
W20260810 01:02:38.089795 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:02:38 dist-test-slave-gp6n krb5kdc[7796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:02:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:02:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 01:02:38.420257 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:35373
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:35373
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:02:38.777221 7810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:38.777518 7810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:38.777612 7810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:38.784943 7810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:02:38.785058 7810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:38.785104 7810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:38.785159 7810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:02:38.785204 7810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:02:38.795176 7810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:35373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:35373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:38.796466 7810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:38.797750 7810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:38.810053 7815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:38.809990 7816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:38.809929 7818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:38.810523 7810 server_base.cc:1034] running on GCE node
I20260810 01:02:38.811774 7810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:38.813263 7810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:38.814525 7810 hybrid_clock.cc:648] HybridClock initialized: now 1786323758814466 us; error 40 us; skew 500 ppm
I20260810 01:02:38.814886 7810 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:02:38 dist-test-slave-gp6n krb5kdc[7796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323758, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:02:38.818163 7810 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 01:02:38.819756 7810 webserver.cc:460] Webserver started at http://127.20.81.190:34303/ using document root <none> and password file <none>
I20260810 01:02:38.820415 7810 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:38.820520 7810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:38.820792 7810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:38.823824 7810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "1ddb2e328f0b4045888663c9b9e736da"
format_stamp: "Formatted at 2026-08-10 01:02:38 on dist-test-slave-gp6n"
server_key: "c670ce0ce324850e900930884306af76"
server_key_iv: "a380a203c7a633fc7e1347676f218507"
server_key_version: "encryptionkey@0"
I20260810 01:02:38.824563 7810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "1ddb2e328f0b4045888663c9b9e736da"
format_stamp: "Formatted at 2026-08-10 01:02:38 on dist-test-slave-gp6n"
server_key: "c670ce0ce324850e900930884306af76"
server_key_iv: "a380a203c7a633fc7e1347676f218507"
server_key_version: "encryptionkey@0"
I20260810 01:02:38.829836 7810 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 01:02:38.833446 7826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:38.834959 7810 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:02:38.835144 7810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "1ddb2e328f0b4045888663c9b9e736da"
format_stamp: "Formatted at 2026-08-10 01:02:38 on dist-test-slave-gp6n"
server_key: "c670ce0ce324850e900930884306af76"
server_key_iv: "a380a203c7a633fc7e1347676f218507"
server_key_version: "encryptionkey@0"
I20260810 01:02:38.835377 7810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:38.872326 7810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:38.877216 7810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:38.877530 7810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:38.897737 7810 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:35373
I20260810 01:02:38.897773 7882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:35373 every 8 connection(s)
I20260810 01:02:38.899204 7810 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:02:38.902596 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 7810
I20260810 01:02:38.902843 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:02:38.903498 7883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:02:38.903702 20806 external_mini_cluster.cc:1428] Setting key ec5ae426c90eaf24ba231aa2692c855c
I20260810 01:02:38.918623 7883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da: Bootstrap starting.
Aug 10 01:02:38 dist-test-slave-gp6n krb5kdc[7796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:02:38.921859 7883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da: Neither blocks nor log segments found. Creating new log.
I20260810 01:02:38.923027 7883 log.cc:824] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:38.925925 7883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da: No bootstrap required, opened a new log
I20260810 01:02:38.928331 7886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:38.905815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41746 (local address 127.20.81.190:35373)
0810 01:02:38.906704 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:38.906715 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:38.906758 (+ 43us) server_negotiation.cc:407] Connection header received
0810 01:02:38.908511 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:38.908532 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:38.908924 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:38.909252 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:38.911128 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:38.912495 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:38.914451 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:38.915220 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:38.915371 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:38.920448 (+ 5077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:38.920456 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:38.920470 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:38.920509 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:38.923788 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:38.924497 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:38.924504 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:38.924510 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:38.924680 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:38.925100 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:38.925105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:38.925114 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:38.925631 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:38.925943 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:38.926473 (+ 530us) server_negotiation.cc:299] Negotiation successful
0810 01:02:38.926947 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":403,"threads_started":1}
I20260810 01:02:38.934285 7883 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddb2e328f0b4045888663c9b9e736da" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 35373 } }
I20260810 01:02:38.934691 7883 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:02:38.934773 7883 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ddb2e328f0b4045888663c9b9e736da, State: Initialized, Role: FOLLOWER
I20260810 01:02:38.935257 7883 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddb2e328f0b4045888663c9b9e736da" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 35373 } }
I20260810 01:02:38.935473 7883 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:02:38.935586 7883 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:02:38.935710 7883 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:02:38.937767 7883 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddb2e328f0b4045888663c9b9e736da" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 35373 } }
I20260810 01:02:38.938269 7883 leader_election.cc:302] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ddb2e328f0b4045888663c9b9e736da; no voters:
I20260810 01:02:38.938947 7883 leader_election.cc:288] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:02:38.939036 7888 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:02:38.939940 7888 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da [term 1 LEADER]: Becoming Leader. State: Replica: 1ddb2e328f0b4045888663c9b9e736da, State: Running, Role: LEADER
I20260810 01:02:38.940521 7888 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddb2e328f0b4045888663c9b9e736da" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 35373 } }
I20260810 01:02:38.941259 7883 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:02:38.943629 7890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ddb2e328f0b4045888663c9b9e736da. Latest consensus state: current_term: 1 leader_uuid: "1ddb2e328f0b4045888663c9b9e736da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddb2e328f0b4045888663c9b9e736da" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 35373 } } }
I20260810 01:02:38.943606 7889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ddb2e328f0b4045888663c9b9e736da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddb2e328f0b4045888663c9b9e736da" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 35373 } } }
I20260810 01:02:38.944056 7890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da [sys.catalog]: This master's current role is: LEADER
I20260810 01:02:38.944077 7889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da [sys.catalog]: This master's current role is: LEADER
I20260810 01:02:38.945513 7895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:02:38.950831 7895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:02:38.960358 7895 catalog_manager.cc:1347] Generated new cluster ID: 822ee59026c644aea8e042b5ad2ce552
I20260810 01:02:38.960508 7895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:02:38.988765 7895 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:02:38.990193 7895 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:02:39.000396 7895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da: Generated new TSK 0
I20260810 01:02:39.001260 7895 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.81.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
I20260810 01:02:39.094085 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:35373
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:02:39.469246 7910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:39.469542 7910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:39.469643 7910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:39.476727 7910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:39.476845 7910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:39.476965 7910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 01:02:39.486825 7910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:35373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:39.488109 7910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:39.489431 7910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:39.502992 7919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:39.503098 7917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:39.503592 7915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:39.503841 7910 server_base.cc:1034] running on GCE node
I20260810 01:02:39.504503 7910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:39.505481 7910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:39.506718 7910 hybrid_clock.cc:648] HybridClock initialized: now 1786323759506675 us; error 52 us; skew 500 ppm
I20260810 01:02:39.507058 7910 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:02:39 dist-test-slave-gp6n krb5kdc[7796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323759, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:02:39.510224 7910 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 01:02:39.511780 7910 webserver.cc:460] Webserver started at http://127.20.81.129:46007/ using document root <none> and password file <none>
I20260810 01:02:39.512511 7910 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:39.512657 7910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:39.512985 7910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:39.516058 7910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "c662d4c515ae49ddaedcea540dc9666a"
format_stamp: "Formatted at 2026-08-10 01:02:39 on dist-test-slave-gp6n"
server_key: "41ac6667d7ad3ff8f2550f7c7b598ec4"
server_key_iv: "ec3bac9909b813cf4f146844689f36ce"
server_key_version: "encryptionkey@0"
I20260810 01:02:39.516839 7910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "c662d4c515ae49ddaedcea540dc9666a"
format_stamp: "Formatted at 2026-08-10 01:02:39 on dist-test-slave-gp6n"
server_key: "41ac6667d7ad3ff8f2550f7c7b598ec4"
server_key_iv: "ec3bac9909b813cf4f146844689f36ce"
server_key_version: "encryptionkey@0"
I20260810 01:02:39.522145 7910 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 01:02:39.525614 7926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:39.527007 7910 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:02:39.527181 7910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "c662d4c515ae49ddaedcea540dc9666a"
format_stamp: "Formatted at 2026-08-10 01:02:39 on dist-test-slave-gp6n"
server_key: "41ac6667d7ad3ff8f2550f7c7b598ec4"
server_key_iv: "ec3bac9909b813cf4f146844689f36ce"
server_key_version: "encryptionkey@0"
I20260810 01:02:39.527401 7910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:39.548269 7910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:39.552737 7910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:39.553040 7910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:39.554031 7910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:02:39.555470 7910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:02:39.555576 7910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:39.555729 7910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:02:39.555804 7910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:39.590091 7910 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:45805
I20260810 01:02:39.590153 8058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:45805 every 8 connection(s)
I20260810 01:02:39.591482 7910 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 01:02:39.593487 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 7910
I20260810 01:02:39.593706 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 01:02:39.594532 20806 external_mini_cluster.cc:1428] Setting key 6b864c4dfd8715d2d87f25565173a4ee
Aug 10 01:02:39 dist-test-slave-gp6n krb5kdc[7796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323759, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:02:39.613914 8061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:39.594466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:53905 (local address 127.20.81.190:35373)
0810 01:02:39.595455 (+ 989us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:39.595483 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:39.596241 (+ 758us) server_negotiation.cc:407] Connection header received
0810 01:02:39.598229 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:39.598238 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:39.598341 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:39.598499 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:39.599952 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:39.601071 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:39.602687 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:39.603380 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:39.603539 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:39.607334 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:39.607344 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:39.607351 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:39.607396 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:39.610650 (+ 3254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:39.611585 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:39.611588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:39.611590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:39.611705 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:39.612209 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:39.612212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:39.612214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:39.612485 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:39.612597 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:39.613464 (+ 867us) server_negotiation.cc:299] Negotiation successful
0810 01:02:39.613655 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":852,"thread_start_us":719,"threads_started":1}
I20260810 01:02:39.615516 8063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:39.595132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:35373 (local address 127.20.81.129:53905)
0810 01:02:39.596063 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:39.596124 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:39.597958 (+ 1834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:39.598687 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:39.598696 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:39.599210 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:02:39.599777 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:39.599795 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:39.601227 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:39.601231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:39.602551 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:39.602563 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:39.603595 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:39.603599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:39.603784 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:39.604485 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:02:39.604512 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:02:39.607113 (+ 2601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:02:39.610786 (+ 3673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:02:39.610795 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:02:39.610814 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:02:39.611467 (+ 653us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:02:39.611824 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:02:39.611831 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:02:39.611838 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:02:39.612094 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:02:39.612612 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:02:39.612622 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:02:39.613069 (+ 447us) client_negotiation.cc:769] Sending connection context
0810 01:02:39.613363 (+ 294us) client_negotiation.cc:240] Negotiation successful
0810 01:02:39.613962 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":371,"threads_started":1}
I20260810 01:02:39.617132 8059 heartbeater.cc:347] Connected to a master server at 127.20.81.190:35373
I20260810 01:02:39.617431 8059 heartbeater.cc:464] Registering TS with master...
I20260810 01:02:39.618150 8059 heartbeater.cc:511] Master 127.20.81.190:35373 requested a full tablet report, sending...
I20260810 01:02:39.620340 7844 ts_manager.cc:194] Registered new tserver with Master: c662d4c515ae49ddaedcea540dc9666a (127.20.81.129:45805)
I20260810 01:02:39.621800 7844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:53905
WARNING: no policy specified for kudu/127.20.81.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
I20260810 01:02:39.667307 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:35373
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:02:40.025107 8066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:40.025408 8066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:40.025610 8066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:40.033210 8066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:40.033329 8066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:40.033557 8066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 01:02:40.044113 8066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:35373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:40.045483 8066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:40.046864 8066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:40.060859 8073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:40.060900 8074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:40.060937 8076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:40.061704 8066 server_base.cc:1034] running on GCE node
I20260810 01:02:40.062381 8066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:40.063427 8066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:40.064666 8066 hybrid_clock.cc:648] HybridClock initialized: now 1786323760064605 us; error 51 us; skew 500 ppm
I20260810 01:02:40.065025 8066 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:02:40 dist-test-slave-gp6n krb5kdc[7796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323760, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:02:40.069160 8066 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 01:02:40.074766 8066 webserver.cc:460] Webserver started at http://127.20.81.130:35337/ using document root <none> and password file <none>
I20260810 01:02:40.075803 8066 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:40.075973 8066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:40.076392 8066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:40.081094 8066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "7b4cde8466cf48778bb06ef2bd003aba"
format_stamp: "Formatted at 2026-08-10 01:02:40 on dist-test-slave-gp6n"
server_key: "b028973dff2fbb4bc0dc464da16a2e5b"
server_key_iv: "10d03b2817c028ec4c89c70fa6c1ec39"
server_key_version: "encryptionkey@0"
I20260810 01:02:40.082202 8066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "7b4cde8466cf48778bb06ef2bd003aba"
format_stamp: "Formatted at 2026-08-10 01:02:40 on dist-test-slave-gp6n"
server_key: "b028973dff2fbb4bc0dc464da16a2e5b"
server_key_iv: "10d03b2817c028ec4c89c70fa6c1ec39"
server_key_version: "encryptionkey@0"
I20260810 01:02:40.089263 8066 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260810 01:02:40.093415 8084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:40.095288 8066 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:02:40.095855 8066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "7b4cde8466cf48778bb06ef2bd003aba"
format_stamp: "Formatted at 2026-08-10 01:02:40 on dist-test-slave-gp6n"
server_key: "b028973dff2fbb4bc0dc464da16a2e5b"
server_key_iv: "10d03b2817c028ec4c89c70fa6c1ec39"
server_key_version: "encryptionkey@0"
I20260810 01:02:40.096167 8066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:40.113459 8066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:40.117969 8066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:40.118278 8066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:40.119287 8066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:02:40.120687 8066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:02:40.120791 8066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:40.120944 8066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:02:40.121021 8066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:40.162498 8066 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:43573
I20260810 01:02:40.162557 8212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:43573 every 8 connection(s)
I20260810 01:02:40.164004 8066 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 01:02:40.170848 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 8066
I20260810 01:02:40.171058 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 01:02:40.171682 20806 external_mini_cluster.cc:1428] Setting key 9a02bd17d5059161eaf66c678b400471
Aug 10 01:02:40 dist-test-slave-gp6n krb5kdc[7796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323760, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:02:40.193336 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:40.166858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:42121 (local address 127.20.81.190:35373)
0810 01:02:40.167351 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:40.167356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:40.168226 (+ 870us) server_negotiation.cc:407] Connection header received
0810 01:02:40.170218 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:40.170224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:40.170314 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:40.170467 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:40.172392 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:40.173410 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:40.175965 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:40.176801 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:40.176964 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:40.182173 (+ 5209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:40.182182 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:40.182189 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:40.182237 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:40.188267 (+ 6030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:40.189585 (+ 1318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:40.189593 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:40.189599 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:40.189766 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:40.190372 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:40.190378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:40.190383 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:40.190996 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:40.191139 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:40.192509 (+ 1370us) server_negotiation.cc:299] Negotiation successful
0810 01:02:40.192789 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":377,"thread_start_us":262,"threads_started":1}
I20260810 01:02:40.194727 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:40.167339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:35373 (local address 127.20.81.130:42121)
0810 01:02:40.168059 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:40.168100 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:40.169909 (+ 1809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:40.170884 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:40.170893 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:40.171533 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:02:40.172177 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:40.172196 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:40.173615 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:40.173623 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:40.175753 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:40.175773 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:40.177289 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:40.177299 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:40.177705 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:40.178585 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:02:40.178623 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:02:40.181855 (+ 3232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:02:40.188468 (+ 6613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:02:40.188482 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:02:40.188504 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:02:40.189218 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:02:40.189908 (+ 690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:02:40.189916 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:02:40.189922 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:02:40.190204 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:02:40.191170 (+ 966us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:02:40.191182 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:02:40.191792 (+ 610us) client_negotiation.cc:769] Sending connection context
0810 01:02:40.192185 (+ 393us) client_negotiation.cc:240] Negotiation successful
0810 01:02:40.192938 (+ 753us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":277,"threads_started":1}
I20260810 01:02:40.196798 8213 heartbeater.cc:347] Connected to a master server at 127.20.81.190:35373
I20260810 01:02:40.197157 8213 heartbeater.cc:464] Registering TS with master...
I20260810 01:02:40.198041 8213 heartbeater.cc:511] Master 127.20.81.190:35373 requested a full tablet report, sending...
I20260810 01:02:40.199928 7844 ts_manager.cc:194] Registered new tserver with Master: 7b4cde8466cf48778bb06ef2bd003aba (127.20.81.130:43573)
I20260810 01:02:40.200796 7844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:42121
WARNING: no policy specified for kudu/127.20.81.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
I20260810 01:02:40.254943 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:35373
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:02:40.611759 8219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:40.612056 8219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:40.612213 8219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:40.619524 8219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:40.619637 8219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:40.619843 8219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 01:02:40.624881 8059 heartbeater.cc:503] Master 127.20.81.190:35373 was elected leader, sending a full tablet report...
I20260810 01:02:40.630450 8219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:35373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:40.631897 8219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:40.633222 8219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:40.647329 8224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:40.647291 8226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:40.647377 8228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:40.648861 8219 server_base.cc:1034] running on GCE node
I20260810 01:02:40.649577 8219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:40.650761 8219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:40.652004 8219 hybrid_clock.cc:648] HybridClock initialized: now 1786323760651959 us; error 38 us; skew 500 ppm
I20260810 01:02:40.652352 8219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:02:40 dist-test-slave-gp6n krb5kdc[7796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323760, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:02:40.661247 8219 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 01:02:40.662909 8219 webserver.cc:460] Webserver started at http://127.20.81.131:41511/ using document root <none> and password file <none>
I20260810 01:02:40.663635 8219 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:40.663787 8219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:40.664119 8219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:40.667313 8219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "35a622f8483f4b3a86317267b05ecbc7"
format_stamp: "Formatted at 2026-08-10 01:02:40 on dist-test-slave-gp6n"
server_key: "4a7b450cc34efcc46a1639968d64eb7d"
server_key_iv: "3a9636e60c6b98ba24a6ab564ad3c681"
server_key_version: "encryptionkey@0"
I20260810 01:02:40.668100 8219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "35a622f8483f4b3a86317267b05ecbc7"
format_stamp: "Formatted at 2026-08-10 01:02:40 on dist-test-slave-gp6n"
server_key: "4a7b450cc34efcc46a1639968d64eb7d"
server_key_iv: "3a9636e60c6b98ba24a6ab564ad3c681"
server_key_version: "encryptionkey@0"
I20260810 01:02:40.673394 8219 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 01:02:40.676995 8236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:40.678553 8219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 01:02:40.678740 8219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "35a622f8483f4b3a86317267b05ecbc7"
format_stamp: "Formatted at 2026-08-10 01:02:40 on dist-test-slave-gp6n"
server_key: "4a7b450cc34efcc46a1639968d64eb7d"
server_key_iv: "3a9636e60c6b98ba24a6ab564ad3c681"
server_key_version: "encryptionkey@0"
I20260810 01:02:40.678947 8219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:40.700963 8219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:40.705298 8219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:40.705593 8219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:40.706629 8219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:02:40.707959 8219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:02:40.708069 8219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:40.708236 8219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:02:40.708312 8219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:40.743592 8219 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:35439
I20260810 01:02:40.743680 8366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:35439 every 8 connection(s)
I20260810 01:02:40.744915 8219 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 01:02:40.749387 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 8219
I20260810 01:02:40.749577 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 01:02:40.750159 20806 external_mini_cluster.cc:1428] Setting key 60516f26e964d6ee403c13bca74ec157
Aug 10 01:02:40 dist-test-slave-gp6n krb5kdc[7796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323760, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:02:40.764830 8369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:40.747508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:39195 (local address 127.20.81.190:35373)
0810 01:02:40.747980 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:40.747985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:40.748947 (+ 962us) server_negotiation.cc:407] Connection header received
0810 01:02:40.751196 (+ 2249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:40.751204 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:40.751294 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:40.751431 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:40.752952 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:40.753735 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:40.755341 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:40.755732 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:40.755869 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:40.759230 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:40.759235 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:40.759238 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:40.759271 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:40.761776 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:40.762515 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:40.762525 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:40.762528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:40.762648 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:40.763080 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:40.763087 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:40.763095 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:40.763347 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:40.763515 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:40.764190 (+ 675us) server_negotiation.cc:299] Negotiation successful
0810 01:02:40.764414 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":252,"threads_started":1}
I20260810 01:02:40.765846 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:40.747982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:35373 (local address 127.20.81.131:39195)
0810 01:02:40.748748 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:40.748797 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:40.750877 (+ 2080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:40.751665 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:40.751673 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:40.752214 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:02:40.752778 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:40.752799 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:40.753882 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:40.753888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:40.755191 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:40.755203 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:40.755961 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:40.755969 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:40.756200 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:40.756821 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:02:40.756843 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:02:40.759023 (+ 2180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:02:40.761914 (+ 2891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:02:40.761924 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:02:40.761941 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:02:40.762347 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:02:40.762762 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:02:40.762766 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:02:40.762770 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:02:40.762955 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:02:40.763480 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:02:40.763487 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:02:40.763806 (+ 319us) client_negotiation.cc:769] Sending connection context
0810 01:02:40.764065 (+ 259us) client_negotiation.cc:240] Negotiation successful
0810 01:02:40.764589 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":273,"threads_started":1}
I20260810 01:02:40.767153 8367 heartbeater.cc:347] Connected to a master server at 127.20.81.190:35373
I20260810 01:02:40.767477 8367 heartbeater.cc:464] Registering TS with master...
I20260810 01:02:40.768189 8367 heartbeater.cc:511] Master 127.20.81.190:35373 requested a full tablet report, sending...
I20260810 01:02:40.769603 7844 ts_manager.cc:194] Registered new tserver with Master: 35a622f8483f4b3a86317267b05ecbc7 (127.20.81.131:35439)
I20260810 01:02:40.770658 7844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:39195
I20260810 01:02:40.777529 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 01:02:40.795006 8369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:40.782217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41762 (local address 127.20.81.190:35373)
0810 01:02:40.782383 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:40.782387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:40.782991 (+ 604us) server_negotiation.cc:407] Connection header received
0810 01:02:40.783145 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:40.783149 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:40.783250 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:40.783342 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:40.784788 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:40.785509 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:40.786883 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:40.787349 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:40.787467 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:40.789891 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:40.789897 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:40.789900 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:40.789933 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:40.792490 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:40.793139 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:40.793143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:40.793145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:40.793277 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:40.793612 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:40.793615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:40.793617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:40.793979 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:40.794174 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:40.794573 (+ 399us) server_negotiation.cc:299] Negotiation successful
0810 01:02:40.794753 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 01:02:40.799885 7844 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 01:02:40.807101 7844 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 01:02:40.831228 8384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:40.819834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50112 (local address 127.20.81.130:43573)
0810 01:02:40.820744 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:40.820754 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:40.821683 (+ 929us) server_negotiation.cc:407] Connection header received
0810 01:02:40.822516 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:40.822525 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:40.822762 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:40.822944 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:40.824142 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:40.826045 (+ 1903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:40.828911 (+ 2866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:40.829889 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:40.830043 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:40.830257 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:40.830654 (+ 397us) server_negotiation.cc:299] Negotiation successful
0810 01:02:40.830870 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":588,"threads_started":1}
I20260810 01:02:40.831346 8385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:40.819523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:43573 (local address 127.0.0.1:50112)
0810 01:02:40.821439 (+ 1916us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:40.821546 (+ 107us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:40.822260 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:40.823319 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:40.823328 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:40.823366 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:40.823853 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:40.823868 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:40.826199 (+ 2331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:40.826204 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:40.828752 (+ 2548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:40.828768 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:40.830102 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:40.830107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:40.830311 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:40.830360 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 01:02:40.830595 (+ 235us) client_negotiation.cc:240] Negotiation successful
0810 01:02:40.830852 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1625,"mutex_wait_us":201,"thread_start_us":1469,"threads_started":1}
I20260810 01:02:40.833206 8387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:40.821867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:45805 (local address 127.0.0.1:45970)
0810 01:02:40.823509 (+ 1642us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:40.823539 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:40.823654 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:40.824677 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:40.824682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:40.824702 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:40.825121 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:40.825136 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:40.826651 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:40.826659 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:40.829882 (+ 3223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:40.829899 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:40.832454 (+ 2555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:40.832461 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:40.832621 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:40.832654 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 01:02:40.832754 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 01:02:40.832877 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1485,"mutex_wait_us":277,"thread_start_us":1314,"threads_started":1}
I20260810 01:02:40.833357 8388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:40.822578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45970 (local address 127.20.81.129:45805)
0810 01:02:40.823854 (+ 1276us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:40.823861 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:40.823883 (+ 22us) server_negotiation.cc:407] Connection header received
0810 01:02:40.823956 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:40.823961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:40.824207 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:40.824399 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:40.825329 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:40.826507 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:40.830090 (+ 3583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:40.832228 (+ 2138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:40.832390 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:40.832647 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:40.832807 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 01:02:40.833005 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1128,"thread_start_us":945,"threads_started":1}
I20260810 01:02:40.835793 8383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:40.820346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:35439 (local address 127.0.0.1:54428)
0810 01:02:40.821439 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:40.821756 (+ 317us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:40.821927 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:40.823810 (+ 1883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:40.823815 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:40.823844 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:40.824346 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:40.824359 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:40.826498 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:40.826508 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:40.829885 (+ 3377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:40.829899 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:40.834955 (+ 5056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:40.834962 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:40.835148 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:40.835182 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 01:02:40.835271 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 01:02:40.835401 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":902,"mutex_wait_us":261,"thread_start_us":1090,"threads_started":1}
I20260810 01:02:40.835842 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:40.821167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54428 (local address 127.20.81.131:35439)
0810 01:02:40.821959 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:40.821971 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:40.822010 (+ 39us) server_negotiation.cc:407] Connection header received
0810 01:02:40.822177 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:40.822186 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:40.822509 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:40.822708 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:40.824587 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:40.826315 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:40.830090 (+ 3775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:40.834754 (+ 4664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:40.834923 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:40.835168 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:40.835311 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 01:02:40.835468 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":803,"threads_started":1}
I20260810 01:02:40.836010 8140 tablet_service.cc:1467] Processing CreateTablet for tablet 22c9bf7339aa4c818c88a5bdd81be899 (DEFAULT_TABLE table=test-table [id=f9d9deb651694b4085703a845c8a4594]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:02:40.837672 8140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22c9bf7339aa4c818c88a5bdd81be899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:02:40.837334 7983 tablet_service.cc:1467] Processing CreateTablet for tablet 22c9bf7339aa4c818c88a5bdd81be899 (DEFAULT_TABLE table=test-table [id=f9d9deb651694b4085703a845c8a4594]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:02:40.838955 7983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22c9bf7339aa4c818c88a5bdd81be899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:02:40.839262 8291 tablet_service.cc:1467] Processing CreateTablet for tablet 22c9bf7339aa4c818c88a5bdd81be899 (DEFAULT_TABLE table=test-table [id=f9d9deb651694b4085703a845c8a4594]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:02:40.840458 8291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22c9bf7339aa4c818c88a5bdd81be899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:02:40.853515 8389 tablet_bootstrap.cc:492] T 22c9bf7339aa4c818c88a5bdd81be899 P 35a622f8483f4b3a86317267b05ecbc7: Bootstrap starting.
I20260810 01:02:40.857012 8389 tablet_bootstrap.cc:654] T 22c9bf7339aa4c818c88a5bdd81be899 P 35a622f8483f4b3a86317267b05ecbc7: Neither blocks nor log segments found. Creating new log.
I20260810 01:02:40.857868 8392 tablet_bootstrap.cc:492] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a: Bootstrap starting.
I20260810 01:02:40.858366 8389 log.cc:824] T 22c9bf7339aa4c818c88a5bdd81be899 P 35a622f8483f4b3a86317267b05ecbc7: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:40.863322 8391 tablet_bootstrap.cc:492] T 22c9bf7339aa4c818c88a5bdd81be899 P 7b4cde8466cf48778bb06ef2bd003aba: Bootstrap starting.
I20260810 01:02:40.862382 8392 tablet_bootstrap.cc:654] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a: Neither blocks nor log segments found. Creating new log.
I20260810 01:02:40.864952 8392 log.cc:824] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:40.866092 8389 tablet_bootstrap.cc:492] T 22c9bf7339aa4c818c88a5bdd81be899 P 35a622f8483f4b3a86317267b05ecbc7: No bootstrap required, opened a new log
I20260810 01:02:40.866441 8389 ts_tablet_manager.cc:1397] T 22c9bf7339aa4c818c88a5bdd81be899 P 35a622f8483f4b3a86317267b05ecbc7: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.003s
I20260810 01:02:40.867026 8391 tablet_bootstrap.cc:654] T 22c9bf7339aa4c818c88a5bdd81be899 P 7b4cde8466cf48778bb06ef2bd003aba: Neither blocks nor log segments found. Creating new log.
I20260810 01:02:40.868243 8392 tablet_bootstrap.cc:492] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a: No bootstrap required, opened a new log
I20260810 01:02:40.868302 8391 log.cc:824] T 22c9bf7339aa4c818c88a5bdd81be899 P 7b4cde8466cf48778bb06ef2bd003aba: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:40.868528 8392 ts_tablet_manager.cc:1397] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260810 01:02:40.871474 8391 tablet_bootstrap.cc:492] T 22c9bf7339aa4c818c88a5bdd81be899 P 7b4cde8466cf48778bb06ef2bd003aba: No bootstrap required, opened a new log
I20260810 01:02:40.871732 8391 ts_tablet_manager.cc:1397] T 22c9bf7339aa4c818c88a5bdd81be899 P 7b4cde8466cf48778bb06ef2bd003aba: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 01:02:40.875830 8389 raft_consensus.cc:348] T 22c9bf7339aa4c818c88a5bdd81be899 P 35a622f8483f4b3a86317267b05ecbc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b4cde8466cf48778bb06ef2bd003aba" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 43573 } } peers { permanent_uuid: "c662d4c515ae49ddaedcea540dc9666a" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45805 } } peers { permanent_uuid: "35a622f8483f4b3a86317267b05ecbc7" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35439 } }
I20260810 01:02:40.876231 8389 raft_consensus.cc:374] T 22c9bf7339aa4c818c88a5bdd81be899 P 35a622f8483f4b3a86317267b05ecbc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:02:40.876375 8389 raft_consensus.cc:729] T 22c9bf7339aa4c818c88a5bdd81be899 P 35a622f8483f4b3a86317267b05ecbc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35a622f8483f4b3a86317267b05ecbc7, State: Initialized, Role: FOLLOWER
I20260810 01:02:40.877048 8389 consensus_queue.cc:260] T 22c9bf7339aa4c818c88a5bdd81be899 P 35a622f8483f4b3a86317267b05ecbc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b4cde8466cf48778bb06ef2bd003aba" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 43573 } } peers { permanent_uuid: "c662d4c515ae49ddaedcea540dc9666a" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45805 } } peers { permanent_uuid: "35a622f8483f4b3a86317267b05ecbc7" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35439 } }
I20260810 01:02:40.878253 8367 heartbeater.cc:503] Master 127.20.81.190:35373 was elected leader, sending a full tablet report...
I20260810 01:02:40.879031 8389 ts_tablet_manager.cc:1428] T 22c9bf7339aa4c818c88a5bdd81be899 P 35a622f8483f4b3a86317267b05ecbc7: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260810 01:02:40.880622 8392 raft_consensus.cc:348] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b4cde8466cf48778bb06ef2bd003aba" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 43573 } } peers { permanent_uuid: "c662d4c515ae49ddaedcea540dc9666a" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45805 } } peers { permanent_uuid: "35a622f8483f4b3a86317267b05ecbc7" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35439 } }
I20260810 01:02:40.881059 8392 raft_consensus.cc:374] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:02:40.881155 8392 raft_consensus.cc:729] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c662d4c515ae49ddaedcea540dc9666a, State: Initialized, Role: FOLLOWER
I20260810 01:02:40.881847 8392 consensus_queue.cc:260] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b4cde8466cf48778bb06ef2bd003aba" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 43573 } } peers { permanent_uuid: "c662d4c515ae49ddaedcea540dc9666a" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45805 } } peers { permanent_uuid: "35a622f8483f4b3a86317267b05ecbc7" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35439 } }
I20260810 01:02:40.882373 8391 raft_consensus.cc:348] T 22c9bf7339aa4c818c88a5bdd81be899 P 7b4cde8466cf48778bb06ef2bd003aba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b4cde8466cf48778bb06ef2bd003aba" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 43573 } } peers { permanent_uuid: "c662d4c515ae49ddaedcea540dc9666a" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45805 } } peers { permanent_uuid: "35a622f8483f4b3a86317267b05ecbc7" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35439 } }
I20260810 01:02:40.882952 8391 raft_consensus.cc:374] T 22c9bf7339aa4c818c88a5bdd81be899 P 7b4cde8466cf48778bb06ef2bd003aba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:02:40.883095 8391 raft_consensus.cc:729] T 22c9bf7339aa4c818c88a5bdd81be899 P 7b4cde8466cf48778bb06ef2bd003aba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b4cde8466cf48778bb06ef2bd003aba, State: Initialized, Role: FOLLOWER
I20260810 01:02:40.883749 8391 consensus_queue.cc:260] T 22c9bf7339aa4c818c88a5bdd81be899 P 7b4cde8466cf48778bb06ef2bd003aba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b4cde8466cf48778bb06ef2bd003aba" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 43573 } } peers { permanent_uuid: "c662d4c515ae49ddaedcea540dc9666a" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45805 } } peers { permanent_uuid: "35a622f8483f4b3a86317267b05ecbc7" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35439 } }
I20260810 01:02:40.886778 8392 ts_tablet_manager.cc:1428] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a: Time spent starting tablet: real 0.018s user 0.010s sys 0.008s
I20260810 01:02:40.886890 8213 heartbeater.cc:503] Master 127.20.81.190:35373 was elected leader, sending a full tablet report...
I20260810 01:02:40.887403 8391 ts_tablet_manager.cc:1428] T 22c9bf7339aa4c818c88a5bdd81be899 P 7b4cde8466cf48778bb06ef2bd003aba: Time spent starting tablet: real 0.016s user 0.011s sys 0.005s
W20260810 01:02:40.916152 8214 tablet.cc:2369] T 22c9bf7339aa4c818c88a5bdd81be899 P 7b4cde8466cf48778bb06ef2bd003aba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 01:02:40.996658 8368 tablet.cc:2369] T 22c9bf7339aa4c818c88a5bdd81be899 P 35a622f8483f4b3a86317267b05ecbc7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:02:41.050625 8397 raft_consensus.cc:482] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:02:41.050997 8397 raft_consensus.cc:504] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b4cde8466cf48778bb06ef2bd003aba" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 43573 } } peers { permanent_uuid: "c662d4c515ae49ddaedcea540dc9666a" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45805 } } peers { permanent_uuid: "35a622f8483f4b3a86317267b05ecbc7" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35439 } }
I20260810 01:02:41.052866 8397 leader_election.cc:288] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b4cde8466cf48778bb06ef2bd003aba (127.20.81.130:43573), 35a622f8483f4b3a86317267b05ecbc7 (127.20.81.131:35439)
I20260810 01:02:41.061429 8384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:41.053393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:60213 (local address 127.20.81.130:43573)
0810 01:02:41.053646 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:41.053653 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:41.054475 (+ 822us) server_negotiation.cc:407] Connection header received
0810 01:02:41.054739 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:41.054746 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:41.054831 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:41.054944 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:41.055752 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:41.056926 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:41.059532 (+ 2606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:41.060363 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:41.060499 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:41.060563 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:41.060997 (+ 434us) server_negotiation.cc:299] Negotiation successful
0810 01:02:41.061153 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260810 01:02:41.061527 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:41.053191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:43573 (local address 127.20.81.129:60213)
0810 01:02:41.054291 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:41.054315 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:41.054555 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:41.054990 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:41.054995 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:41.055079 (+ 84us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:41.055584 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:41.055603 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:41.057087 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:41.057092 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:41.059342 (+ 2250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:41.059359 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:41.060597 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:41.060603 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:41.060803 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:41.060845 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 01:02:41.060939 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 01:02:41.061037 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":934,"mutex_wait_us":23,"spinlock_wait_cycles":25344,"thread_start_us":584,"threads_started":1}
I20260810 01:02:41.061645 8400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:41.053315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:35439 (local address 127.20.81.129:42075)
0810 01:02:41.054552 (+ 1237us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:41.054568 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:41.054678 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:41.054990 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:41.054995 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:41.055033 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:41.055607 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:41.055616 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:41.057086 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:41.057092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:41.059511 (+ 2419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:41.059528 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:41.060804 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:41.060808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:41.060963 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:41.060995 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 01:02:41.061096 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 01:02:41.061220 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1109,"mutex_wait_us":186,"thread_start_us":848,"threads_started":1}
I20260810 01:02:41.061957 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:41.053480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:42075 (local address 127.20.81.131:35439)
0810 01:02:41.053732 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:41.053739 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:41.054659 (+ 920us) server_negotiation.cc:407] Connection header received
0810 01:02:41.054805 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:41.054810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:41.054890 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:41.054976 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:41.055752 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:41.056917 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:41.059678 (+ 2761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:41.060580 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:41.060753 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:41.060826 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:41.061147 (+ 321us) server_negotiation.cc:299] Negotiation successful
0810 01:02:41.061302 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260810 01:02:41.062789 8162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22c9bf7339aa4c818c88a5bdd81be899" candidate_uuid: "c662d4c515ae49ddaedcea540dc9666a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b4cde8466cf48778bb06ef2bd003aba" is_pre_election: true
I20260810 01:02:41.062901 8316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22c9bf7339aa4c818c88a5bdd81be899" candidate_uuid: "c662d4c515ae49ddaedcea540dc9666a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35a622f8483f4b3a86317267b05ecbc7" is_pre_election: true
I20260810 01:02:41.063378 8162 raft_consensus.cc:2454] T 22c9bf7339aa4c818c88a5bdd81be899 P 7b4cde8466cf48778bb06ef2bd003aba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c662d4c515ae49ddaedcea540dc9666a in term 0.
I20260810 01:02:41.063445 8316 raft_consensus.cc:2454] T 22c9bf7339aa4c818c88a5bdd81be899 P 35a622f8483f4b3a86317267b05ecbc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c662d4c515ae49ddaedcea540dc9666a in term 0.
I20260810 01:02:41.064152 7929 leader_election.cc:302] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 35a622f8483f4b3a86317267b05ecbc7, 7b4cde8466cf48778bb06ef2bd003aba, c662d4c515ae49ddaedcea540dc9666a; no voters:
I20260810 01:02:41.064575 8397 raft_consensus.cc:2781] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:02:41.064725 8397 raft_consensus.cc:482] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:02:41.064870 8397 raft_consensus.cc:3037] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:02:41.067299 8397 raft_consensus.cc:504] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b4cde8466cf48778bb06ef2bd003aba" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 43573 } } peers { permanent_uuid: "c662d4c515ae49ddaedcea540dc9666a" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45805 } } peers { permanent_uuid: "35a622f8483f4b3a86317267b05ecbc7" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35439 } }
I20260810 01:02:41.067962 8397 leader_election.cc:288] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a [CANDIDATE]: Term 1 election: Requested vote from peers 7b4cde8466cf48778bb06ef2bd003aba (127.20.81.130:43573), 35a622f8483f4b3a86317267b05ecbc7 (127.20.81.131:35439)
I20260810 01:02:41.068490 8316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22c9bf7339aa4c818c88a5bdd81be899" candidate_uuid: "c662d4c515ae49ddaedcea540dc9666a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35a622f8483f4b3a86317267b05ecbc7"
I20260810 01:02:41.068509 8162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22c9bf7339aa4c818c88a5bdd81be899" candidate_uuid: "c662d4c515ae49ddaedcea540dc9666a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b4cde8466cf48778bb06ef2bd003aba"
I20260810 01:02:41.068706 8316 raft_consensus.cc:3037] T 22c9bf7339aa4c818c88a5bdd81be899 P 35a622f8483f4b3a86317267b05ecbc7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:02:41.068719 8162 raft_consensus.cc:3037] T 22c9bf7339aa4c818c88a5bdd81be899 P 7b4cde8466cf48778bb06ef2bd003aba [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:02:41.071067 8316 raft_consensus.cc:2454] T 22c9bf7339aa4c818c88a5bdd81be899 P 35a622f8483f4b3a86317267b05ecbc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c662d4c515ae49ddaedcea540dc9666a in term 1.
I20260810 01:02:41.071069 8162 raft_consensus.cc:2454] T 22c9bf7339aa4c818c88a5bdd81be899 P 7b4cde8466cf48778bb06ef2bd003aba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c662d4c515ae49ddaedcea540dc9666a in term 1.
I20260810 01:02:41.071542 7927 leader_election.cc:302] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35a622f8483f4b3a86317267b05ecbc7, c662d4c515ae49ddaedcea540dc9666a; no voters:
I20260810 01:02:41.071897 8397 raft_consensus.cc:2781] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:02:41.072346 8397 raft_consensus.cc:686] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a [term 1 LEADER]: Becoming Leader. State: Replica: c662d4c515ae49ddaedcea540dc9666a, State: Running, Role: LEADER
I20260810 01:02:41.072839 8397 consensus_queue.cc:237] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b4cde8466cf48778bb06ef2bd003aba" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 43573 } } peers { permanent_uuid: "c662d4c515ae49ddaedcea540dc9666a" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45805 } } peers { permanent_uuid: "35a622f8483f4b3a86317267b05ecbc7" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35439 } }
I20260810 01:02:41.077888 7844 catalog_manager.cc:5495] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a reported cstate change: term changed from 0 to 1, leader changed from <none> to c662d4c515ae49ddaedcea540dc9666a (127.20.81.129). New cstate: current_term: 1 leader_uuid: "c662d4c515ae49ddaedcea540dc9666a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b4cde8466cf48778bb06ef2bd003aba" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 43573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c662d4c515ae49ddaedcea540dc9666a" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35a622f8483f4b3a86317267b05ecbc7" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35439 } health_report { overall_health: UNKNOWN } } }
W20260810 01:02:41.094115 8060 tablet.cc:2369] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:02:41.129580 8388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:41.123709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45976 (local address 127.20.81.129:45805)
0810 01:02:41.123942 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:41.123949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:41.124057 (+ 108us) server_negotiation.cc:407] Connection header received
0810 01:02:41.124312 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:41.124316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:41.124397 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:41.124557 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:02:41.124962 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:41.125695 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:41.127302 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:41.127716 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:41.127853 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:41.128392 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:02:41.128931 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:02:41.129119 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:41.129171 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 01:02:41.129311 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 01:02:41.138823 8316 raft_consensus.cc:1261] T 22c9bf7339aa4c818c88a5bdd81be899 P 35a622f8483f4b3a86317267b05ecbc7 [term 1 FOLLOWER]: Refusing update from remote peer c662d4c515ae49ddaedcea540dc9666a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:02:41.138792 8162 raft_consensus.cc:1261] T 22c9bf7339aa4c818c88a5bdd81be899 P 7b4cde8466cf48778bb06ef2bd003aba [term 1 FOLLOWER]: Refusing update from remote peer c662d4c515ae49ddaedcea540dc9666a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:02:41.139917 8401 consensus_queue.cc:1035] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b4cde8466cf48778bb06ef2bd003aba" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 43573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:02:41.140353 8397 consensus_queue.cc:1035] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a [LEADER]: Connected to new peer: Peer: permanent_uuid: "35a622f8483f4b3a86317267b05ecbc7" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 35439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:02:41.151821 8405 mvcc.cc:205] Tried to move back new op lower bound from 7316782125613973504 to 7316782125357957120. Current Snapshot: MvccSnapshot[applied={T|T < 7316782125613973504}]
I20260810 01:02:41.157042 8406 mvcc.cc:205] Tried to move back new op lower bound from 7316782125613973504 to 7316782125357957120. Current Snapshot: MvccSnapshot[applied={T|T < 7316782125613973504}]
I20260810 01:02:41.176169 7843 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:02:41.177178 7843 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:02:41.182379 7843 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ddb2e328f0b4045888663c9b9e736da: Sending DeleteTablet for 3 replicas of tablet 22c9bf7339aa4c818c88a5bdd81be899
I20260810 01:02:41.183820 8291 tablet_service.cc:1514] Processing DeleteTablet for tablet 22c9bf7339aa4c818c88a5bdd81be899 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:02:41 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:54428
I20260810 01:02:41.183956 8140 tablet_service.cc:1514] Processing DeleteTablet for tablet 22c9bf7339aa4c818c88a5bdd81be899 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:02:41 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:50112
I20260810 01:02:41.184226 7983 tablet_service.cc:1514] Processing DeleteTablet for tablet 22c9bf7339aa4c818c88a5bdd81be899 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:02:41 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:45970
I20260810 01:02:41.186852 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 7910
Aug 10 01:02:41 dist-test-slave-gp6n krb5kdc[7796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.129@KRBTEST.COM
I20260810 01:02:41.208321 8420 tablet_replica.cc:330] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a: stopping tablet replica
I20260810 01:02:41.208884 8420 raft_consensus.cc:2229] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a [term 1 LEADER]: Raft consensus shutting down.
I20260810 01:02:41.209734 8420 raft_consensus.cc:2258] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:02:41.210963 8419 tablet_replica.cc:330] T 22c9bf7339aa4c818c88a5bdd81be899 P 7b4cde8466cf48778bb06ef2bd003aba: stopping tablet replica
I20260810 01:02:41.211396 8419 raft_consensus.cc:2229] T 22c9bf7339aa4c818c88a5bdd81be899 P 7b4cde8466cf48778bb06ef2bd003aba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:02:41.211926 8419 raft_consensus.cc:2258] T 22c9bf7339aa4c818c88a5bdd81be899 P 7b4cde8466cf48778bb06ef2bd003aba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:02:41.212692 8418 tablet_replica.cc:330] T 22c9bf7339aa4c818c88a5bdd81be899 P 35a622f8483f4b3a86317267b05ecbc7: stopping tablet replica
I20260810 01:02:41.213119 8418 raft_consensus.cc:2229] T 22c9bf7339aa4c818c88a5bdd81be899 P 35a622f8483f4b3a86317267b05ecbc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:02:41.213176 8420 ts_tablet_manager.cc:1905] T 22c9bf7339aa4c818c88a5bdd81be899 P c662d4c515ae49ddaedcea540dc9666a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:02:41.213627 8418 raft_consensus.cc:2258] T 22c9bf7339aa4c818c88a5bdd81be899 P 35a622f8483f4b3a86317267b05ecbc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:02:41.216169 8388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:41.189993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45980 (local address 127.20.81.129:45805)
0810 01:02:41.190199 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:41.190205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:41.192737 (+ 2532us) server_negotiation.cc:407] Connection header received
0810 01:02:41.193550 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:41.193556 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:41.193635 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:41.193701 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:41.195793 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:41.197279 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:41.199728 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:41.200671 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:41.200786 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:41.207198 (+ 6412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:41.207218 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:41.207243 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:41.207315 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:41.212455 (+ 5140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:41.213414 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:41.213424 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:41.213427 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:41.213616 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:41.214119 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:41.214125 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:41.214132 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:41.214606 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:41.214900 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:41.215513 (+ 613us) server_negotiation.cc:299] Negotiation successful
0810 01:02:41.215816 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 01:02:41.217648 8053 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:41.222970 8418 ts_tablet_manager.cc:1905] T 22c9bf7339aa4c818c88a5bdd81be899 P 35a622f8483f4b3a86317267b05ecbc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:02:41.224040 8419 ts_tablet_manager.cc:1905] T 22c9bf7339aa4c818c88a5bdd81be899 P 7b4cde8466cf48778bb06ef2bd003aba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:02:41.228641 8419 ts_tablet_manager.cc:1918] T 22c9bf7339aa4c818c88a5bdd81be899 P 7b4cde8466cf48778bb06ef2bd003aba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:02:41.228637 8418 ts_tablet_manager.cc:1918] T 22c9bf7339aa4c818c88a5bdd81be899 P 35a622f8483f4b3a86317267b05ecbc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:02:41.228829 8418 log.cc:1196] T 22c9bf7339aa4c818c88a5bdd81be899 P 35a622f8483f4b3a86317267b05ecbc7: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/22c9bf7339aa4c818c88a5bdd81be899
I20260810 01:02:41.228818 8419 log.cc:1196] T 22c9bf7339aa4c818c88a5bdd81be899 P 7b4cde8466cf48778bb06ef2bd003aba: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/22c9bf7339aa4c818c88a5bdd81be899
I20260810 01:02:41.229354 8418 ts_tablet_manager.cc:1939] T 22c9bf7339aa4c818c88a5bdd81be899 P 35a622f8483f4b3a86317267b05ecbc7: Deleting consensus metadata
I20260810 01:02:41.229388 8419 ts_tablet_manager.cc:1939] T 22c9bf7339aa4c818c88a5bdd81be899 P 7b4cde8466cf48778bb06ef2bd003aba: Deleting consensus metadata
I20260810 01:02:41.231130 7828 catalog_manager.cc:4841] TS 7b4cde8466cf48778bb06ef2bd003aba (127.20.81.130:43573): tablet 22c9bf7339aa4c818c88a5bdd81be899 (table test-table [id=f9d9deb651694b4085703a845c8a4594]) successfully deleted
I20260810 01:02:41.231199 7827 catalog_manager.cc:4841] TS 35a622f8483f4b3a86317267b05ecbc7 (127.20.81.131:35439): tablet 22c9bf7339aa4c818c88a5bdd81be899 (table test-table [id=f9d9deb651694b4085703a845c8a4594]) successfully deleted
I20260810 01:02:41.323684 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 7910
I20260810 01:02:41.346832 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 8066
W20260810 01:02:41.346866 7828 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.81.129:45805 (error 108)
I20260810 01:02:41.348901 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:41.348196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:45805 (local address 127.0.0.1:45992)
0810 01:02:41.348629 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:41.348750 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:45805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":227,"threads_started":1}
W20260810 01:02:41.349115 7828 catalog_manager.cc:4584] TS c662d4c515ae49ddaedcea540dc9666a (127.20.81.129:45805): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 22c9bf7339aa4c818c88a5bdd81be899: Network error: Client connection negotiation failed: client connection to 127.20.81.129:45805: connect: Connection refused (error 111)
Aug 10 01:02:41 dist-test-slave-gp6n krb5kdc[7796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.130@KRBTEST.COM
I20260810 01:02:41.363098 8384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:41.347734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50114 (local address 127.20.81.130:43573)
0810 01:02:41.347937 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:41.347944 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:41.348124 (+ 180us) server_negotiation.cc:407] Connection header received
0810 01:02:41.348303 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:41.348307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:41.348384 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:41.348514 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:41.350044 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:41.350913 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:41.352259 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:41.352649 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:41.352785 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:41.357182 (+ 4397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:41.357196 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:41.357210 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:41.357250 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:41.360166 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:41.360857 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:41.360864 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:41.360867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:41.361002 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:41.361376 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:41.361382 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:41.361386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:41.361779 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:41.362113 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:41.362395 (+ 282us) server_negotiation.cc:299] Negotiation successful
0810 01:02:41.362814 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 01:02:41.363727 8208 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:41.402616 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:41.402093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:45805 (local address 127.0.0.1:41688)
0810 01:02:41.402308 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:41.402448 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:45805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260810 01:02:41.403754 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:41.403367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:45805 (local address 127.0.0.1:41702)
0810 01:02:41.403529 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:41.403610 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:45805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 01:02:41.456125 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 8066
I20260810 01:02:41.464531 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:41.464071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:45805 (local address 127.0.0.1:41718)
0810 01:02:41.464281 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:41.464378 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:45805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 01:02:41.465749 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:41.465352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:45805 (local address 127.0.0.1:41724)
0810 01:02:41.465524 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:41.465599 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:45805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 01:02:41.478451 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 8219
Aug 10 01:02:41 dist-test-slave-gp6n krb5kdc[7796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.131@KRBTEST.COM
I20260810 01:02:41.495138 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:41.479241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51264 (local address 127.20.81.131:35439)
0810 01:02:41.479425 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:41.479434 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:41.479519 (+ 85us) server_negotiation.cc:407] Connection header received
0810 01:02:41.479741 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:41.479745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:41.479811 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:41.479931 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:41.481373 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:41.482245 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:41.483958 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:41.484370 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:41.484511 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:41.489260 (+ 4749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:41.489289 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:41.489306 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:41.489349 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:41.492238 (+ 2889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:41.492952 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:41.492959 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:41.492961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:41.493116 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:41.493498 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:41.493504 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:41.493507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:41.493870 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:41.494181 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:41.494550 (+ 369us) server_negotiation.cc:299] Negotiation successful
0810 01:02:41.494777 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 01:02:41.495841 8362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:41.546533 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:41.545953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:45805 (local address 127.0.0.1:41736)
0810 01:02:41.546199 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:41.546290 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:45805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260810 01:02:41.547675 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:41.547270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:45805 (local address 127.0.0.1:41748)
0810 01:02:41.547476 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:41.547545 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:45805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260810 01:02:41.588510 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 8219
I20260810 01:02:41.609066 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 7810
I20260810 01:02:41.610271 7877 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:41.722126 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:41.721698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:45805 (local address 127.0.0.1:41752)
0810 01:02:41.721884 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:41.721976 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:45805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 01:02:41.722105 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 7810
2026-08-10T01:02:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4863 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T01:02:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:02:41Z Disabled control of system clock
I20260810 01:02:41.823769 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:37635
--webserver_interface=127.20.81.190
--webserver_port=0
--builtin_ntp_servers=127.20.81.148:38213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:37635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 01:02:42.178591 8431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:42.178885 8431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:42.179046 8431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:42.186697 8431 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:02:42.186806 8431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:42.186846 8431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:42.186952 8431 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:02:42.187034 8431 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:02:42.196497 8431 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:38213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:37635
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:37635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:42.197809 8431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:42.199683 8431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:42.211798 8438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:42.211900 8441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:42.212185 8437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:42.212623 8431 server_base.cc:1034] running on GCE node
I20260810 01:02:42.213418 8431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:42.214830 8431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:42.216068 8431 hybrid_clock.cc:648] HybridClock initialized: now 1786323762215998 us; error 40 us; skew 500 ppm
I20260810 01:02:42.216475 8431 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:02:42.220151 8431 webserver.cc:460] Webserver started at http://127.20.81.190:39417/ using document root <none> and password file <none>
I20260810 01:02:42.221051 8431 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:42.221179 8431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:42.221582 8431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:42.225673 8431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "28471d3da72a4bccbb4093f0729d1e0c"
format_stamp: "Formatted at 2026-08-10 01:02:42 on dist-test-slave-gp6n"
server_key: "997ddb208cb6a08d0da42d15507689d4"
server_key_iv: "4a4b4ec95a247f7d40ed79068ee9bffa"
server_key_version: "encryptionkey@0"
I20260810 01:02:42.226691 8431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "28471d3da72a4bccbb4093f0729d1e0c"
format_stamp: "Formatted at 2026-08-10 01:02:42 on dist-test-slave-gp6n"
server_key: "997ddb208cb6a08d0da42d15507689d4"
server_key_iv: "4a4b4ec95a247f7d40ed79068ee9bffa"
server_key_version: "encryptionkey@0"
I20260810 01:02:42.232070 8431 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 01:02:42.235616 8447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:42.236996 8431 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 01:02:42.237181 8431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "28471d3da72a4bccbb4093f0729d1e0c"
format_stamp: "Formatted at 2026-08-10 01:02:42 on dist-test-slave-gp6n"
server_key: "997ddb208cb6a08d0da42d15507689d4"
server_key_iv: "4a4b4ec95a247f7d40ed79068ee9bffa"
server_key_version: "encryptionkey@0"
I20260810 01:02:42.237406 8431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:42.261443 8431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:42.262334 8431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:42.262691 8431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:42.281231 8431 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:37635
I20260810 01:02:42.281268 8500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:37635 every 8 connection(s)
I20260810 01:02:42.282649 8431 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:02:42.283210 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 8431
I20260810 01:02:42.283418 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:02:42.284122 20806 external_mini_cluster.cc:1428] Setting key b357f10aa69c8aa7278e073f7a5ca3fe
I20260810 01:02:42.287329 8501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:02:42.298008 8504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:42.286136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41748 (local address 127.20.81.190:37635)
0810 01:02:42.287093 (+ 957us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:42.287104 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:42.287147 (+ 43us) server_negotiation.cc:407] Connection header received
0810 01:02:42.289226 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:42.289248 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:42.289734 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:42.290153 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:42.290778 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:42.292105 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:42.293559 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:42.294237 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:42.294467 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:42.295674 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:42.295681 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:42.295694 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:02:42.296282 (+ 588us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:42.296376 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:02:42.296397 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:42.296553 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:42.296779 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 01:02:42.297085 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":354,"threads_started":1}
I20260810 01:02:42.298710 8501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c: Bootstrap starting.
I20260810 01:02:42.302024 8501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c: Neither blocks nor log segments found. Creating new log.
I20260810 01:02:42.303088 8501 log.cc:824] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:42.305846 8501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c: No bootstrap required, opened a new log
I20260810 01:02:42.313081 8501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28471d3da72a4bccbb4093f0729d1e0c" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 37635 } }
I20260810 01:02:42.313488 8501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:02:42.313592 8501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28471d3da72a4bccbb4093f0729d1e0c, State: Initialized, Role: FOLLOWER
I20260810 01:02:42.314235 8501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28471d3da72a4bccbb4093f0729d1e0c" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 37635 } }
I20260810 01:02:42.314502 8501 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:02:42.314646 8501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:02:42.314844 8501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:02:42.317022 8501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28471d3da72a4bccbb4093f0729d1e0c" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 37635 } }
I20260810 01:02:42.317569 8501 leader_election.cc:302] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28471d3da72a4bccbb4093f0729d1e0c; no voters:
I20260810 01:02:42.318241 8501 leader_election.cc:288] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:02:42.318392 8506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:02:42.319188 8506 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c [term 1 LEADER]: Becoming Leader. State: Replica: 28471d3da72a4bccbb4093f0729d1e0c, State: Running, Role: LEADER
I20260810 01:02:42.319774 8506 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28471d3da72a4bccbb4093f0729d1e0c" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 37635 } }
I20260810 01:02:42.320762 8501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:02:42.322681 8508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28471d3da72a4bccbb4093f0729d1e0c. Latest consensus state: current_term: 1 leader_uuid: "28471d3da72a4bccbb4093f0729d1e0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28471d3da72a4bccbb4093f0729d1e0c" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 37635 } } }
I20260810 01:02:42.322993 8507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28471d3da72a4bccbb4093f0729d1e0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28471d3da72a4bccbb4093f0729d1e0c" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 37635 } } }
I20260810 01:02:42.323216 8508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c [sys.catalog]: This master's current role is: LEADER
I20260810 01:02:42.323279 8507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c [sys.catalog]: This master's current role is: LEADER
I20260810 01:02:42.325407 8515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:02:42.330641 8515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:02:42.340052 8515 catalog_manager.cc:1347] Generated new cluster ID: de6a1f4b74a64360878600847cc80bb1
I20260810 01:02:42.340184 8515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:02:42.354805 8515 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:02:42.356097 8515 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:02:42.363163 8515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c: Generated new TSK 0
I20260810 01:02:42.363996 8515 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:02:42.371881 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:37635
--builtin_ntp_servers=127.20.81.148:38213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 01:02:42.731145 8528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:42.731472 8528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:42.731639 8528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:42.738889 8528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:42.739034 8528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:42.739207 8528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 01:02:42.748447 8528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:38213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:37635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:42.749797 8528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:42.751641 8528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:42.767088 8534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:42.767159 8535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:42.767196 8537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:42.768325 8528 server_base.cc:1034] running on GCE node
I20260810 01:02:42.769070 8528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:42.770107 8528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:42.771420 8528 hybrid_clock.cc:648] HybridClock initialized: now 1786323762771353 us; error 45 us; skew 500 ppm
I20260810 01:02:42.771776 8528 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:02:42.774120 8528 webserver.cc:460] Webserver started at http://127.20.81.129:40203/ using document root <none> and password file <none>
I20260810 01:02:42.774887 8528 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:42.775029 8528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:42.775372 8528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:42.778232 8528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "a6b3752c4a9f42bc9f374bbab4597a0d"
format_stamp: "Formatted at 2026-08-10 01:02:42 on dist-test-slave-gp6n"
server_key: "e14cf6939ca24781cf6f7ba3b3388fd1"
server_key_iv: "1a613c8c9d34120980c912d7d7e37125"
server_key_version: "encryptionkey@0"
I20260810 01:02:42.779145 8528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "a6b3752c4a9f42bc9f374bbab4597a0d"
format_stamp: "Formatted at 2026-08-10 01:02:42 on dist-test-slave-gp6n"
server_key: "e14cf6939ca24781cf6f7ba3b3388fd1"
server_key_iv: "1a613c8c9d34120980c912d7d7e37125"
server_key_version: "encryptionkey@0"
I20260810 01:02:42.784198 8528 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 01:02:42.787845 8545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:42.789196 8528 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 01:02:42.789431 8528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "a6b3752c4a9f42bc9f374bbab4597a0d"
format_stamp: "Formatted at 2026-08-10 01:02:42 on dist-test-slave-gp6n"
server_key: "e14cf6939ca24781cf6f7ba3b3388fd1"
server_key_iv: "1a613c8c9d34120980c912d7d7e37125"
server_key_version: "encryptionkey@0"
I20260810 01:02:42.789652 8528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:42.806538 8528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:42.807327 8528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:42.807631 8528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:42.808570 8528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:02:42.810047 8528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:02:42.810174 8528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:42.810312 8528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:02:42.810401 8528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:42.844718 8528 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:44133
I20260810 01:02:42.844777 8670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:44133 every 8 connection(s)
I20260810 01:02:42.846076 8528 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 01:02:42.849989 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 8528
I20260810 01:02:42.850178 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 01:02:42.850816 20806 external_mini_cluster.cc:1428] Setting key cb66dcb9b6886dabe54551899912a5fb
I20260810 01:02:42.853526 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:37635
--builtin_ntp_servers=127.20.81.148:38213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 01:02:42.859061 8673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:42.848778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:40077 (local address 127.20.81.190:37635)
0810 01:02:42.849199 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:42.849204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:42.850734 (+ 1530us) server_negotiation.cc:407] Connection header received
0810 01:02:42.852552 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:42.852557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:42.852646 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:42.852760 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:42.853907 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:42.854685 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:42.856228 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:42.856647 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:42.856777 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:42.858151 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:42.858156 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:42.858159 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:02:42.858176 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:42.858216 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:02:42.858227 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:42.858377 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:42.858625 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 01:02:42.858725 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":218,"threads_started":1}
I20260810 01:02:42.860044 8674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:42.849455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:37635 (local address 127.20.81.129:40077)
0810 01:02:42.850585 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:42.850622 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:42.852284 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:42.853011 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:42.853018 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:42.853104 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:02:42.853728 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:42.853751 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:42.854832 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:42.854837 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:42.856082 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:42.856094 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:42.856885 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:42.856895 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:42.857123 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:42.857845 (+ 722us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 01:02:42.857847 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:02:42.857883 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 01:02:42.857955 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:02:42.858387 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:02:42.858414 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:02:42.858425 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 01:02:42.858609 (+ 184us) client_negotiation.cc:240] Negotiation successful
0810 01:02:42.859000 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":324,"threads_started":1}
I20260810 01:02:42.861533 8671 heartbeater.cc:347] Connected to a master server at 127.20.81.190:37635
I20260810 01:02:42.861829 8671 heartbeater.cc:464] Registering TS with master...
I20260810 01:02:42.862648 8671 heartbeater.cc:511] Master 127.20.81.190:37635 requested a full tablet report, sending...
I20260810 01:02:42.864850 8465 ts_manager.cc:194] Registered new tserver with Master: a6b3752c4a9f42bc9f374bbab4597a0d (127.20.81.129:44133)
I20260810 01:02:42.866879 8465 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.81.129:40077
W20260810 01:02:43.214609 8675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:43.214879 8675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:43.215042 8675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:43.222389 8675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:43.222541 8675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:43.222730 8675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 01:02:43.232148 8675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:38213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:37635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:43.233493 8675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:43.235375 8675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:43.248754 8682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:43.248804 8685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:43.249083 8683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:43.249797 8675 server_base.cc:1034] running on GCE node
I20260810 01:02:43.250488 8675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:43.251468 8675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:43.252719 8675 hybrid_clock.cc:648] HybridClock initialized: now 1786323763252674 us; error 37 us; skew 500 ppm
I20260810 01:02:43.253072 8675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:02:43.255535 8675 webserver.cc:460] Webserver started at http://127.20.81.130:40145/ using document root <none> and password file <none>
I20260810 01:02:43.256254 8675 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:43.256392 8675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:43.256716 8675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:43.259694 8675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "28b5aee175544b56bbae84bf17ec430c"
format_stamp: "Formatted at 2026-08-10 01:02:43 on dist-test-slave-gp6n"
server_key: "ba20b273f88079d1509bee38f8c37a51"
server_key_iv: "a58f70eacce0dcae7b8baeb5688f393d"
server_key_version: "encryptionkey@0"
I20260810 01:02:43.260527 8675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "28b5aee175544b56bbae84bf17ec430c"
format_stamp: "Formatted at 2026-08-10 01:02:43 on dist-test-slave-gp6n"
server_key: "ba20b273f88079d1509bee38f8c37a51"
server_key_iv: "a58f70eacce0dcae7b8baeb5688f393d"
server_key_version: "encryptionkey@0"
I20260810 01:02:43.265672 8675 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 01:02:43.269292 8691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:43.270710 8675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:02:43.270905 8675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "28b5aee175544b56bbae84bf17ec430c"
format_stamp: "Formatted at 2026-08-10 01:02:43 on dist-test-slave-gp6n"
server_key: "ba20b273f88079d1509bee38f8c37a51"
server_key_iv: "a58f70eacce0dcae7b8baeb5688f393d"
server_key_version: "encryptionkey@0"
I20260810 01:02:43.271122 8675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:43.285749 8675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:43.286949 8675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:43.287506 8675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:43.288478 8675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:02:43.289947 8675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:02:43.290074 8675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:43.290210 8675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:02:43.290308 8675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:43.326093 8675 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:34471
I20260810 01:02:43.326195 8816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:34471 every 8 connection(s)
I20260810 01:02:43.327505 8675 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 01:02:43.337850 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 8675
I20260810 01:02:43.338059 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 01:02:43.338596 20806 external_mini_cluster.cc:1428] Setting key 900a9859d2aa53fb7ab1c412d2e9507b
I20260810 01:02:43.340618 8673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:43.330137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:52333 (local address 127.20.81.190:37635)
0810 01:02:43.330327 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:43.330334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:43.331872 (+ 1538us) server_negotiation.cc:407] Connection header received
0810 01:02:43.333659 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:43.333664 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:43.333756 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:43.333879 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:43.335002 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:43.335729 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:43.337268 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:43.337876 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:43.338017 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:43.339671 (+ 1654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:43.339677 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:43.339682 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:02:43.339704 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:43.339753 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:02:43.339770 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:43.339866 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:43.340098 (+ 232us) server_negotiation.cc:299] Negotiation successful
0810 01:02:43.340188 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 01:02:43.341502 8819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:43.330753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:37635 (local address 127.20.81.130:52333)
0810 01:02:43.331692 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:43.331737 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:43.333397 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:43.334099 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:43.334107 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:43.334193 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:02:43.334837 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:43.334854 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:43.335876 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:43.335883 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:43.337127 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:43.337139 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:43.338160 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:43.338170 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:43.338458 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:43.339258 (+ 800us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 01:02:43.339261 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:02:43.339298 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 01:02:43.339431 (+ 133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:02:43.339888 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:02:43.339897 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:02:43.339903 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 01:02:43.340032 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 01:02:43.340306 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":298,"threads_started":1}
I20260810 01:02:43.342010 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:37635
--builtin_ntp_servers=127.20.81.148:38213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 01:02:43.342960 8817 heartbeater.cc:347] Connected to a master server at 127.20.81.190:37635
I20260810 01:02:43.343247 8817 heartbeater.cc:464] Registering TS with master...
I20260810 01:02:43.343967 8817 heartbeater.cc:511] Master 127.20.81.190:37635 requested a full tablet report, sending...
I20260810 01:02:43.345398 8465 ts_manager.cc:194] Registered new tserver with Master: 28b5aee175544b56bbae84bf17ec430c (127.20.81.130:34471)
I20260810 01:02:43.346626 8465 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.81.130:52333
W20260810 01:02:43.689980 8820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:43.690382 8820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:43.690575 8820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:43.701300 8820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:43.701404 8820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:43.701512 8820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 01:02:43.711875 8820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:38213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:37635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:43.713199 8820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:43.715055 8820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:43.728493 8825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:43.728533 8826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:43.728473 8828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:43.729418 8820 server_base.cc:1034] running on GCE node
I20260810 01:02:43.730118 8820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:43.731190 8820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:43.732432 8820 hybrid_clock.cc:648] HybridClock initialized: now 1786323763732385 us; error 35 us; skew 500 ppm
I20260810 01:02:43.732755 8820 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:02:43.735239 8820 webserver.cc:460] Webserver started at http://127.20.81.131:40245/ using document root <none> and password file <none>
I20260810 01:02:43.735975 8820 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:43.736114 8820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:43.736440 8820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:43.739459 8820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "38c50e46f6e4432993b9d21dfaa15bf4"
format_stamp: "Formatted at 2026-08-10 01:02:43 on dist-test-slave-gp6n"
server_key: "237e3621c64bb12ecc4db0a43b5e57ce"
server_key_iv: "1ff90915b7c2954dc28c7b93b40a2b3f"
server_key_version: "encryptionkey@0"
I20260810 01:02:43.740312 8820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "38c50e46f6e4432993b9d21dfaa15bf4"
format_stamp: "Formatted at 2026-08-10 01:02:43 on dist-test-slave-gp6n"
server_key: "237e3621c64bb12ecc4db0a43b5e57ce"
server_key_iv: "1ff90915b7c2954dc28c7b93b40a2b3f"
server_key_version: "encryptionkey@0"
I20260810 01:02:43.745709 8820 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260810 01:02:43.749249 8835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:43.750658 8820 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:02:43.750836 8820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "38c50e46f6e4432993b9d21dfaa15bf4"
format_stamp: "Formatted at 2026-08-10 01:02:43 on dist-test-slave-gp6n"
server_key: "237e3621c64bb12ecc4db0a43b5e57ce"
server_key_iv: "1ff90915b7c2954dc28c7b93b40a2b3f"
server_key_version: "encryptionkey@0"
I20260810 01:02:43.751073 8820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:43.766821 8820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:43.767702 8820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:43.768002 8820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:43.769023 8820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:02:43.770524 8820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:02:43.770613 8820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:43.770695 8820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:02:43.770750 8820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:43.805402 8820 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:46185
I20260810 01:02:43.805516 8955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:46185 every 8 connection(s)
I20260810 01:02:43.806919 8820 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 01:02:43.815610 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 8820
I20260810 01:02:43.815828 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 01:02:43.816497 20806 external_mini_cluster.cc:1428] Setting key 09541c0bec619b04e6679a8e11747de4
I20260810 01:02:43.821162 8673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:43.809697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:55621 (local address 127.20.81.190:37635)
0810 01:02:43.809902 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:43.809907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:43.811460 (+ 1553us) server_negotiation.cc:407] Connection header received
0810 01:02:43.813510 (+ 2050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:43.813516 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:43.813621 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:43.813749 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:43.815010 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:43.816320 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:43.818261 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:43.818979 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:43.819128 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:43.820467 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:43.820472 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:43.820476 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:02:43.820494 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:43.820530 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:02:43.820543 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:43.820623 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:43.820832 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 01:02:43.820913 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 01:02:43.822141 8958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:43.810245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:37635 (local address 127.20.81.131:55621)
0810 01:02:43.811259 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:43.811315 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:43.813194 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:43.814013 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:43.814022 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:43.814117 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:02:43.814826 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:43.814844 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:43.816515 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:43.816519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:43.818098 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:43.818111 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:43.819187 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:43.819192 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:43.819387 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:43.820173 (+ 786us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 01:02:43.820176 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:02:43.820213 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 01:02:43.820287 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:02:43.820633 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:02:43.820640 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:02:43.820646 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 01:02:43.820787 (+ 141us) client_negotiation.cc:240] Negotiation successful
0810 01:02:43.821079 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":317,"threads_started":1}
I20260810 01:02:43.823671 8956 heartbeater.cc:347] Connected to a master server at 127.20.81.190:37635
I20260810 01:02:43.823998 8956 heartbeater.cc:464] Registering TS with master...
I20260810 01:02:43.824745 8956 heartbeater.cc:511] Master 127.20.81.190:37635 requested a full tablet report, sending...
I20260810 01:02:43.826221 8465 ts_manager.cc:194] Registered new tserver with Master: 38c50e46f6e4432993b9d21dfaa15bf4 (127.20.81.131:46185)
I20260810 01:02:43.827169 8465 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.81.131:55621
I20260810 01:02:43.832023 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 01:02:43.842177 8673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:43.836263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41762 (local address 127.20.81.190:37635)
0810 01:02:43.836529 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:43.836535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:43.836849 (+ 314us) server_negotiation.cc:407] Connection header received
0810 01:02:43.836966 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:43.836969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:43.837042 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:43.837129 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:43.837605 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:43.838269 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:43.839743 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:43.840134 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:43.840228 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:43.841391 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:43.841396 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:43.841400 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:02:43.841417 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:43.841447 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:02:43.841460 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:43.841531 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:43.841739 (+ 208us) server_negotiation.cc:299] Negotiation successful
0810 01:02:43.841855 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 01:02:43.846472 8465 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 01:02:43.853353 8465 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 01:02:43.870126 8671 heartbeater.cc:503] Master 127.20.81.190:37635 was elected leader, sending a full tablet report...
I20260810 01:02:43.882508 8973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:43.867540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:44133 (local address 127.0.0.1:43514)
0810 01:02:43.868845 (+ 1305us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:43.868880 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:43.869028 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:43.870940 (+ 1912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:43.870948 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:43.870994 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:43.871434 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:43.871448 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:43.874237 (+ 2789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:43.874246 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:43.877569 (+ 3323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:43.877592 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:43.881431 (+ 3839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:43.881438 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:43.881682 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:43.881769 (+ 87us) client_negotiation.cc:769] Sending connection context
0810 01:02:43.881896 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 01:02:43.882121 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1119,"thread_start_us":930,"threads_started":1}
I20260810 01:02:43.883749 8972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:43.866962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43514 (local address 127.20.81.129:44133)
0810 01:02:43.869688 (+ 2726us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:43.869700 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:43.869738 (+ 38us) server_negotiation.cc:407] Connection header received
0810 01:02:43.869881 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:43.869893 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:43.870218 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:43.871643 (+ 1425us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:43.871916 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:43.874036 (+ 2120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:43.877928 (+ 3892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:43.880088 (+ 2160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:43.882857 (+ 2769us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:43.883144 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:43.883214 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 01:02:43.883396 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2496,"thread_start_us":2352,"threads_started":1}
I20260810 01:02:43.888526 8970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:43.866344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:34471 (local address 127.0.0.1:43322)
0810 01:02:43.867256 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:43.867319 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:43.867545 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:43.881638 (+ 14093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:43.881643 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:43.881668 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:43.882082 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:43.882102 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:43.884260 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:43.884269 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:43.886062 (+ 1793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:43.886073 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:43.887616 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:43.887625 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:43.887866 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:43.887905 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 01:02:43.888011 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 01:02:43.888172 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":419,"threads_started":1}
I20260810 01:02:43.888780 8971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:43.866673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43322 (local address 127.20.81.130:34471)
0810 01:02:43.880665 (+ 13992us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:43.880676 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:43.880714 (+ 38us) server_negotiation.cc:407] Connection header received
0810 01:02:43.880838 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:43.880847 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:43.881166 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:43.881388 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:43.882299 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:43.884124 (+ 1825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:43.886244 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:43.887248 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:43.887507 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:43.887764 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:43.888079 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 01:02:43.888248 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13833,"thread_start_us":13701,"threads_started":1}
I20260810 01:02:43.888352 8601 tablet_service.cc:1467] Processing CreateTablet for tablet b61f6d557f5341a39a402ae96083bfe0 (DEFAULT_TABLE table=test-table [id=8563787628ef44ea90f1f22755086df3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:02:43.890208 8601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b61f6d557f5341a39a402ae96083bfe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:02:43.892298 8975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:43.869606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:46185 (local address 127.0.0.1:54094)
0810 01:02:43.881033 (+ 11427us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:43.881053 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:43.881172 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:43.883703 (+ 2531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:43.883708 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:43.883737 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:43.885112 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:43.885128 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:43.887277 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:43.887282 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:43.889430 (+ 2148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:43.889444 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:43.891589 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:43.891596 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:43.891767 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:43.891799 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 01:02:43.891883 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 01:02:43.892012 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11305,"thread_start_us":11174,"threads_started":1}
I20260810 01:02:43.892468 8976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:43.870853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54094 (local address 127.20.81.131:46185)
0810 01:02:43.883174 (+ 12321us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:43.883182 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:43.883210 (+ 28us) server_negotiation.cc:407] Connection header received
0810 01:02:43.883304 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:43.883309 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:43.883485 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:43.883710 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:43.885364 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:43.887112 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:43.889644 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:43.891376 (+ 1732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:43.891533 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:43.891834 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:43.891926 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 01:02:43.892093 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12174,"thread_start_us":11673,"threads_started":1}
I20260810 01:02:43.892894 8745 tablet_service.cc:1467] Processing CreateTablet for tablet b61f6d557f5341a39a402ae96083bfe0 (DEFAULT_TABLE table=test-table [id=8563787628ef44ea90f1f22755086df3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:02:43.894199 8745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b61f6d557f5341a39a402ae96083bfe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:02:43.896378 8886 tablet_service.cc:1467] Processing CreateTablet for tablet b61f6d557f5341a39a402ae96083bfe0 (DEFAULT_TABLE table=test-table [id=8563787628ef44ea90f1f22755086df3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:02:43.897593 8886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b61f6d557f5341a39a402ae96083bfe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:02:43.905475 8977 tablet_bootstrap.cc:492] T b61f6d557f5341a39a402ae96083bfe0 P 28b5aee175544b56bbae84bf17ec430c: Bootstrap starting.
I20260810 01:02:43.909242 8977 tablet_bootstrap.cc:654] T b61f6d557f5341a39a402ae96083bfe0 P 28b5aee175544b56bbae84bf17ec430c: Neither blocks nor log segments found. Creating new log.
I20260810 01:02:43.910279 8977 log.cc:824] T b61f6d557f5341a39a402ae96083bfe0 P 28b5aee175544b56bbae84bf17ec430c: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:43.912513 8979 tablet_bootstrap.cc:492] T b61f6d557f5341a39a402ae96083bfe0 P a6b3752c4a9f42bc9f374bbab4597a0d: Bootstrap starting.
I20260810 01:02:43.913525 8977 tablet_bootstrap.cc:492] T b61f6d557f5341a39a402ae96083bfe0 P 28b5aee175544b56bbae84bf17ec430c: No bootstrap required, opened a new log
I20260810 01:02:43.913723 8980 tablet_bootstrap.cc:492] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4: Bootstrap starting.
I20260810 01:02:43.913825 8977 ts_tablet_manager.cc:1397] T b61f6d557f5341a39a402ae96083bfe0 P 28b5aee175544b56bbae84bf17ec430c: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 01:02:43.916129 8979 tablet_bootstrap.cc:654] T b61f6d557f5341a39a402ae96083bfe0 P a6b3752c4a9f42bc9f374bbab4597a0d: Neither blocks nor log segments found. Creating new log.
I20260810 01:02:43.917009 8980 tablet_bootstrap.cc:654] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4: Neither blocks nor log segments found. Creating new log.
I20260810 01:02:43.917618 8979 log.cc:824] T b61f6d557f5341a39a402ae96083bfe0 P a6b3752c4a9f42bc9f374bbab4597a0d: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:43.918351 8980 log.cc:824] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:43.920945 8979 tablet_bootstrap.cc:492] T b61f6d557f5341a39a402ae96083bfe0 P a6b3752c4a9f42bc9f374bbab4597a0d: No bootstrap required, opened a new log
I20260810 01:02:43.921226 8979 ts_tablet_manager.cc:1397] T b61f6d557f5341a39a402ae96083bfe0 P a6b3752c4a9f42bc9f374bbab4597a0d: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260810 01:02:43.923226 8980 tablet_bootstrap.cc:492] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4: No bootstrap required, opened a new log
I20260810 01:02:43.923522 8980 ts_tablet_manager.cc:1397] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260810 01:02:43.924588 8977 raft_consensus.cc:348] T b61f6d557f5341a39a402ae96083bfe0 P 28b5aee175544b56bbae84bf17ec430c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b5aee175544b56bbae84bf17ec430c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 34471 } } peers { permanent_uuid: "a6b3752c4a9f42bc9f374bbab4597a0d" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 44133 } } peers { permanent_uuid: "38c50e46f6e4432993b9d21dfaa15bf4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 46185 } }
I20260810 01:02:43.925066 8977 raft_consensus.cc:374] T b61f6d557f5341a39a402ae96083bfe0 P 28b5aee175544b56bbae84bf17ec430c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:02:43.925176 8977 raft_consensus.cc:729] T b61f6d557f5341a39a402ae96083bfe0 P 28b5aee175544b56bbae84bf17ec430c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28b5aee175544b56bbae84bf17ec430c, State: Initialized, Role: FOLLOWER
I20260810 01:02:43.925809 8977 consensus_queue.cc:260] T b61f6d557f5341a39a402ae96083bfe0 P 28b5aee175544b56bbae84bf17ec430c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b5aee175544b56bbae84bf17ec430c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 34471 } } peers { permanent_uuid: "a6b3752c4a9f42bc9f374bbab4597a0d" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 44133 } } peers { permanent_uuid: "38c50e46f6e4432993b9d21dfaa15bf4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 46185 } }
I20260810 01:02:43.927101 8817 heartbeater.cc:503] Master 127.20.81.190:37635 was elected leader, sending a full tablet report...
I20260810 01:02:43.927708 8977 ts_tablet_manager.cc:1428] T b61f6d557f5341a39a402ae96083bfe0 P 28b5aee175544b56bbae84bf17ec430c: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260810 01:02:43.931895 8979 raft_consensus.cc:348] T b61f6d557f5341a39a402ae96083bfe0 P a6b3752c4a9f42bc9f374bbab4597a0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b5aee175544b56bbae84bf17ec430c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 34471 } } peers { permanent_uuid: "a6b3752c4a9f42bc9f374bbab4597a0d" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 44133 } } peers { permanent_uuid: "38c50e46f6e4432993b9d21dfaa15bf4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 46185 } }
I20260810 01:02:43.932287 8979 raft_consensus.cc:374] T b61f6d557f5341a39a402ae96083bfe0 P a6b3752c4a9f42bc9f374bbab4597a0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:02:43.932411 8979 raft_consensus.cc:729] T b61f6d557f5341a39a402ae96083bfe0 P a6b3752c4a9f42bc9f374bbab4597a0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6b3752c4a9f42bc9f374bbab4597a0d, State: Initialized, Role: FOLLOWER
I20260810 01:02:43.933117 8979 consensus_queue.cc:260] T b61f6d557f5341a39a402ae96083bfe0 P a6b3752c4a9f42bc9f374bbab4597a0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b5aee175544b56bbae84bf17ec430c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 34471 } } peers { permanent_uuid: "a6b3752c4a9f42bc9f374bbab4597a0d" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 44133 } } peers { permanent_uuid: "38c50e46f6e4432993b9d21dfaa15bf4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 46185 } }
I20260810 01:02:43.933398 8980 raft_consensus.cc:348] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b5aee175544b56bbae84bf17ec430c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 34471 } } peers { permanent_uuid: "a6b3752c4a9f42bc9f374bbab4597a0d" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 44133 } } peers { permanent_uuid: "38c50e46f6e4432993b9d21dfaa15bf4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 46185 } }
I20260810 01:02:43.933900 8980 raft_consensus.cc:374] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:02:43.934036 8980 raft_consensus.cc:729] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c50e46f6e4432993b9d21dfaa15bf4, State: Initialized, Role: FOLLOWER
I20260810 01:02:43.934716 8980 consensus_queue.cc:260] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b5aee175544b56bbae84bf17ec430c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 34471 } } peers { permanent_uuid: "a6b3752c4a9f42bc9f374bbab4597a0d" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 44133 } } peers { permanent_uuid: "38c50e46f6e4432993b9d21dfaa15bf4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 46185 } }
I20260810 01:02:43.935566 8979 ts_tablet_manager.cc:1428] T b61f6d557f5341a39a402ae96083bfe0 P a6b3752c4a9f42bc9f374bbab4597a0d: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260810 01:02:43.936008 8956 heartbeater.cc:503] Master 127.20.81.190:37635 was elected leader, sending a full tablet report...
I20260810 01:02:43.936691 8980 ts_tablet_manager.cc:1428] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
W20260810 01:02:44.058718 8957 tablet.cc:2369] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 01:02:44.079658 8818 tablet.cc:2369] T b61f6d557f5341a39a402ae96083bfe0 P 28b5aee175544b56bbae84bf17ec430c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 01:02:44.098701 8672 tablet.cc:2369] T b61f6d557f5341a39a402ae96083bfe0 P a6b3752c4a9f42bc9f374bbab4597a0d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:02:44.267148 8985 raft_consensus.cc:482] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:02:44.267491 8985 raft_consensus.cc:504] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b5aee175544b56bbae84bf17ec430c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 34471 } } peers { permanent_uuid: "a6b3752c4a9f42bc9f374bbab4597a0d" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 44133 } } peers { permanent_uuid: "38c50e46f6e4432993b9d21dfaa15bf4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 46185 } }
I20260810 01:02:44.269239 8985 leader_election.cc:288] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28b5aee175544b56bbae84bf17ec430c (127.20.81.130:34471), a6b3752c4a9f42bc9f374bbab4597a0d (127.20.81.129:44133)
I20260810 01:02:44.276820 8972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:44.269854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:46157 (local address 127.20.81.129:44133)
0810 01:02:44.270106 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:44.270115 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:44.270145 (+ 30us) server_negotiation.cc:407] Connection header received
0810 01:02:44.270230 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:44.270236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:44.270331 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:44.270479 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:44.270933 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:44.271944 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:44.274369 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:44.275709 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:44.275895 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:44.275983 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:44.276413 (+ 430us) server_negotiation.cc:299] Negotiation successful
0810 01:02:44.276561 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 01:02:44.276865 8958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:44.269617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:44133 (local address 127.20.81.131:46157)
0810 01:02:44.269783 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:44.269805 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:44.269950 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:44.270517 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:44.270522 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:44.270551 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:44.270771 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:44.270782 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:44.272095 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:44.272098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:44.274173 (+ 2075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:44.274187 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:44.275883 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:44.275888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:44.276224 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:44.276269 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 01:02:44.276372 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 01:02:44.276531 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":39}
I20260810 01:02:44.276914 8971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:44.269687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:50063 (local address 127.20.81.130:34471)
0810 01:02:44.269905 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:44.269911 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:44.270551 (+ 640us) server_negotiation.cc:407] Connection header received
0810 01:02:44.270756 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:44.270760 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:44.270831 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:44.270946 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:44.271408 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:44.272294 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:44.274651 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:44.275884 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:44.276023 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:44.276083 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:44.276533 (+ 450us) server_negotiation.cc:299] Negotiation successful
0810 01:02:44.276673 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 01:02:44.277204 8986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:44.269622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:34471 (local address 127.20.81.131:50063)
0810 01:02:44.270388 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:44.270450 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:44.270622 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:44.270975 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:44.270978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:44.270997 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:44.271254 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:44.271261 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:44.272438 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:44.272443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:44.274480 (+ 2037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:44.274491 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:44.276129 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:44.276135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:44.276369 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:44.276398 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 01:02:44.276475 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 01:02:44.276575 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":199,"spinlock_wait_cycles":21760,"thread_start_us":320,"threads_started":1}
I20260810 01:02:44.277966 8625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b61f6d557f5341a39a402ae96083bfe0" candidate_uuid: "38c50e46f6e4432993b9d21dfaa15bf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6b3752c4a9f42bc9f374bbab4597a0d" is_pre_election: true
I20260810 01:02:44.278261 8767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b61f6d557f5341a39a402ae96083bfe0" candidate_uuid: "38c50e46f6e4432993b9d21dfaa15bf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28b5aee175544b56bbae84bf17ec430c" is_pre_election: true
I20260810 01:02:44.278527 8625 raft_consensus.cc:2454] T b61f6d557f5341a39a402ae96083bfe0 P a6b3752c4a9f42bc9f374bbab4597a0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c50e46f6e4432993b9d21dfaa15bf4 in term 0.
I20260810 01:02:44.278837 8767 raft_consensus.cc:2454] T b61f6d557f5341a39a402ae96083bfe0 P 28b5aee175544b56bbae84bf17ec430c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c50e46f6e4432993b9d21dfaa15bf4 in term 0.
I20260810 01:02:44.279167 8839 leader_election.cc:302] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c50e46f6e4432993b9d21dfaa15bf4, a6b3752c4a9f42bc9f374bbab4597a0d; no voters:
I20260810 01:02:44.279563 8985 raft_consensus.cc:2781] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:02:44.279709 8985 raft_consensus.cc:482] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:02:44.279815 8985 raft_consensus.cc:3037] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:02:44.282092 8985 raft_consensus.cc:504] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b5aee175544b56bbae84bf17ec430c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 34471 } } peers { permanent_uuid: "a6b3752c4a9f42bc9f374bbab4597a0d" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 44133 } } peers { permanent_uuid: "38c50e46f6e4432993b9d21dfaa15bf4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 46185 } }
I20260810 01:02:44.282775 8985 leader_election.cc:288] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4 [CANDIDATE]: Term 1 election: Requested vote from peers 28b5aee175544b56bbae84bf17ec430c (127.20.81.130:34471), a6b3752c4a9f42bc9f374bbab4597a0d (127.20.81.129:44133)
I20260810 01:02:44.283262 8767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b61f6d557f5341a39a402ae96083bfe0" candidate_uuid: "38c50e46f6e4432993b9d21dfaa15bf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28b5aee175544b56bbae84bf17ec430c"
I20260810 01:02:44.283380 8625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b61f6d557f5341a39a402ae96083bfe0" candidate_uuid: "38c50e46f6e4432993b9d21dfaa15bf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6b3752c4a9f42bc9f374bbab4597a0d"
I20260810 01:02:44.283510 8767 raft_consensus.cc:3037] T b61f6d557f5341a39a402ae96083bfe0 P 28b5aee175544b56bbae84bf17ec430c [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:02:44.283589 8625 raft_consensus.cc:3037] T b61f6d557f5341a39a402ae96083bfe0 P a6b3752c4a9f42bc9f374bbab4597a0d [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:02:44.286851 8767 raft_consensus.cc:2454] T b61f6d557f5341a39a402ae96083bfe0 P 28b5aee175544b56bbae84bf17ec430c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c50e46f6e4432993b9d21dfaa15bf4 in term 1.
I20260810 01:02:44.286851 8625 raft_consensus.cc:2454] T b61f6d557f5341a39a402ae96083bfe0 P a6b3752c4a9f42bc9f374bbab4597a0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c50e46f6e4432993b9d21dfaa15bf4 in term 1.
I20260810 01:02:44.287310 8837 leader_election.cc:302] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28b5aee175544b56bbae84bf17ec430c, 38c50e46f6e4432993b9d21dfaa15bf4; no voters:
I20260810 01:02:44.287662 8985 raft_consensus.cc:2781] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:02:44.288122 8985 raft_consensus.cc:686] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4 [term 1 LEADER]: Becoming Leader. State: Replica: 38c50e46f6e4432993b9d21dfaa15bf4, State: Running, Role: LEADER
I20260810 01:02:44.288714 8985 consensus_queue.cc:237] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b5aee175544b56bbae84bf17ec430c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 34471 } } peers { permanent_uuid: "a6b3752c4a9f42bc9f374bbab4597a0d" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 44133 } } peers { permanent_uuid: "38c50e46f6e4432993b9d21dfaa15bf4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 46185 } }
I20260810 01:02:44.294291 8464 catalog_manager.cc:5495] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c50e46f6e4432993b9d21dfaa15bf4 (127.20.81.131). New cstate: current_term: 1 leader_uuid: "38c50e46f6e4432993b9d21dfaa15bf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b5aee175544b56bbae84bf17ec430c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 34471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6b3752c4a9f42bc9f374bbab4597a0d" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 44133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c50e46f6e4432993b9d21dfaa15bf4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 46185 } health_report { overall_health: HEALTHY } } }
I20260810 01:02:44.346338 8976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:44.340639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54110 (local address 127.20.81.131:46185)
0810 01:02:44.340869 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:44.340877 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:44.341032 (+ 155us) server_negotiation.cc:407] Connection header received
0810 01:02:44.341199 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:44.341203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:44.341293 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:44.341476 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:02:44.341824 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:44.342685 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:44.344149 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:44.344579 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:44.344712 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:44.345227 (+ 515us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:02:44.345732 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:02:44.345891 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:44.345997 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 01:02:44.346073 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 01:02:44.354228 8625 raft_consensus.cc:1261] T b61f6d557f5341a39a402ae96083bfe0 P a6b3752c4a9f42bc9f374bbab4597a0d [term 1 FOLLOWER]: Refusing update from remote peer 38c50e46f6e4432993b9d21dfaa15bf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:02:44.354282 8767 raft_consensus.cc:1261] T b61f6d557f5341a39a402ae96083bfe0 P 28b5aee175544b56bbae84bf17ec430c [term 1 FOLLOWER]: Refusing update from remote peer 38c50e46f6e4432993b9d21dfaa15bf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:02:44.355310 8987 consensus_queue.cc:1035] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28b5aee175544b56bbae84bf17ec430c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 34471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:02:44.355674 8985 consensus_queue.cc:1035] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6b3752c4a9f42bc9f374bbab4597a0d" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 44133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:02:44.364369 8990 mvcc.cc:205] Tried to move back new op lower bound from 7316782138785738752 to 7316782138531377152. Current Snapshot: MvccSnapshot[applied={T|T < 7316782138785738752}]
I20260810 01:02:44.379962 8465 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:02:44.380264 8465 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:02:44.383881 8465 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28471d3da72a4bccbb4093f0729d1e0c: Sending DeleteTablet for 3 replicas of tablet b61f6d557f5341a39a402ae96083bfe0
I20260810 01:02:44.385167 8601 tablet_service.cc:1514] Processing DeleteTablet for tablet b61f6d557f5341a39a402ae96083bfe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:02:44 UTC) from {username='slave'} at 127.0.0.1:43514
I20260810 01:02:44.385167 8745 tablet_service.cc:1514] Processing DeleteTablet for tablet b61f6d557f5341a39a402ae96083bfe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:02:44 UTC) from {username='slave'} at 127.0.0.1:43322
I20260810 01:02:44.386821 8886 tablet_service.cc:1514] Processing DeleteTablet for tablet b61f6d557f5341a39a402ae96083bfe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:02:44 UTC) from {username='slave'} at 127.0.0.1:54094
I20260810 01:02:44.388128 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 8528
I20260810 01:02:44.396519 9001 tablet_replica.cc:330] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4: stopping tablet replica
I20260810 01:02:44.397115 9001 raft_consensus.cc:2229] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4 [term 1 LEADER]: Raft consensus shutting down.
I20260810 01:02:44.398010 9001 raft_consensus.cc:2258] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:02:44.399922 9000 tablet_replica.cc:330] T b61f6d557f5341a39a402ae96083bfe0 P a6b3752c4a9f42bc9f374bbab4597a0d: stopping tablet replica
I20260810 01:02:44.400484 9000 raft_consensus.cc:2229] T b61f6d557f5341a39a402ae96083bfe0 P a6b3752c4a9f42bc9f374bbab4597a0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:02:44.401114 9000 raft_consensus.cc:2258] T b61f6d557f5341a39a402ae96083bfe0 P a6b3752c4a9f42bc9f374bbab4597a0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:02:44.403791 9000 ts_tablet_manager.cc:1905] T b61f6d557f5341a39a402ae96083bfe0 P a6b3752c4a9f42bc9f374bbab4597a0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:02:44.405664 9001 ts_tablet_manager.cc:1905] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:02:44.410866 8999 tablet_replica.cc:330] T b61f6d557f5341a39a402ae96083bfe0 P 28b5aee175544b56bbae84bf17ec430c: stopping tablet replica
I20260810 01:02:44.411342 8999 raft_consensus.cc:2229] T b61f6d557f5341a39a402ae96083bfe0 P 28b5aee175544b56bbae84bf17ec430c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:02:44.411913 8999 raft_consensus.cc:2258] T b61f6d557f5341a39a402ae96083bfe0 P 28b5aee175544b56bbae84bf17ec430c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:02:44.413605 8972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:44.391143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43524 (local address 127.20.81.129:44133)
0810 01:02:44.391332 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:44.391338 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:44.402095 (+ 10757us) server_negotiation.cc:407] Connection header received
0810 01:02:44.402558 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:44.402567 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:44.402706 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:44.402806 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:44.403678 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:44.405187 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:44.408113 (+ 2926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:44.409082 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:44.409209 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:44.411604 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:44.411615 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:44.411631 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:02:44.412400 (+ 769us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:44.412451 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:02:44.412473 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:44.412550 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:44.413062 (+ 512us) server_negotiation.cc:299] Negotiation successful
0810 01:02:44.413198 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 01:02:44.414779 8665 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:44.415763 8999 ts_tablet_manager.cc:1905] T b61f6d557f5341a39a402ae96083bfe0 P 28b5aee175544b56bbae84bf17ec430c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:02:44.419703 9001 ts_tablet_manager.cc:1918] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:02:44.419974 9001 log.cc:1196] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/b61f6d557f5341a39a402ae96083bfe0
I20260810 01:02:44.420640 9001 ts_tablet_manager.cc:1939] T b61f6d557f5341a39a402ae96083bfe0 P 38c50e46f6e4432993b9d21dfaa15bf4: Deleting consensus metadata
I20260810 01:02:44.422901 8448 catalog_manager.cc:4841] TS 38c50e46f6e4432993b9d21dfaa15bf4 (127.20.81.131:46185): tablet b61f6d557f5341a39a402ae96083bfe0 (table test-table [id=8563787628ef44ea90f1f22755086df3]) successfully deleted
I20260810 01:02:44.424567 8999 ts_tablet_manager.cc:1918] T b61f6d557f5341a39a402ae96083bfe0 P 28b5aee175544b56bbae84bf17ec430c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:02:44.424747 8999 log.cc:1196] T b61f6d557f5341a39a402ae96083bfe0 P 28b5aee175544b56bbae84bf17ec430c: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/b61f6d557f5341a39a402ae96083bfe0
I20260810 01:02:44.425225 8999 ts_tablet_manager.cc:1939] T b61f6d557f5341a39a402ae96083bfe0 P 28b5aee175544b56bbae84bf17ec430c: Deleting consensus metadata
I20260810 01:02:44.427024 8449 catalog_manager.cc:4841] TS 28b5aee175544b56bbae84bf17ec430c (127.20.81.130:34471): tablet b61f6d557f5341a39a402ae96083bfe0 (table test-table [id=8563787628ef44ea90f1f22755086df3]) successfully deleted
I20260810 01:02:44.543777 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 8528
W20260810 01:02:44.572499 8451 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.81.129:44133 (error 108)
I20260810 01:02:44.573346 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 8675
I20260810 01:02:44.575109 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:44.573983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:44133 (local address 127.0.0.1:43532)
0810 01:02:44.574642 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:44.574838 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:44133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":615,"threads_started":1}
W20260810 01:02:44.575489 8451 catalog_manager.cc:4584] TS a6b3752c4a9f42bc9f374bbab4597a0d (127.20.81.129:44133): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b61f6d557f5341a39a402ae96083bfe0: Network error: Client connection negotiation failed: client connection to 127.20.81.129:44133: connect: Connection refused (error 111)
I20260810 01:02:44.581736 8971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:44.574196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43338 (local address 127.20.81.130:34471)
0810 01:02:44.574466 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:44.574475 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:44.574500 (+ 25us) server_negotiation.cc:407] Connection header received
0810 01:02:44.574647 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:44.574656 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:44.574779 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:44.574900 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:44.575393 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:44.576398 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:44.578352 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:44.578902 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:44.579051 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:44.580222 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:44.580232 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:44.580246 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:02:44.580909 (+ 663us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:44.580959 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:02:44.580981 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:44.581076 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:44.581229 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 01:02:44.581324 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 01:02:44.582616 8812 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:44.629230 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:44.628666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:44133 (local address 127.0.0.1:43536)
0810 01:02:44.628923 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:44.629052 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:44133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260810 01:02:44.630507 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:44.630040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:44133 (local address 127.0.0.1:43546)
0810 01:02:44.630211 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:44.630305 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:44133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260810 01:02:44.677367 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 8675
I20260810 01:02:44.691542 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:44.690892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:44133 (local address 127.0.0.1:43552)
0810 01:02:44.691191 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:44.691318 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:44133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260810 01:02:44.692884 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:44.692404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:44133 (local address 127.0.0.1:43568)
0810 01:02:44.692593 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:44.692687 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:44133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 01:02:44.699378 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 8820
I20260810 01:02:44.706883 8976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:44.700384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54114 (local address 127.20.81.131:46185)
0810 01:02:44.700581 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:44.700588 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:44.700611 (+ 23us) server_negotiation.cc:407] Connection header received
0810 01:02:44.700687 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:44.700693 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:44.700784 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:44.700909 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:44.701336 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:44.702125 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:44.703527 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:44.703995 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:44.704143 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:44.705258 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:44.705267 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:44.705282 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:02:44.705918 (+ 636us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:44.705977 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:02:44.706005 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:44.706131 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:44.706277 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 01:02:44.706387 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 01:02:44.707857 8951 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:44.773725 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:44.773169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:44133 (local address 127.0.0.1:43580)
0810 01:02:44.773466 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:44.773559 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:44133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260810 01:02:44.775200 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:44.774719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:44133 (local address 127.0.0.1:43592)
0810 01:02:44.774965 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:44.775054 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:44133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260810 01:02:44.831337 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 8820
I20260810 01:02:44.856401 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 8431
I20260810 01:02:44.857707 8496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:44.969887 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:44.969118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:44133 (local address 127.0.0.1:43598)
0810 01:02:44.969547 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:44.969694 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:44133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218}
I20260810 01:02:44.969919 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 8431
2026-08-10T01:02:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3242 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:02:45 dist-test-slave-gp6n krb5kdc[9013](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:02:45 dist-test-slave-gp6n krb5kdc[9013](info): set up 2 sockets
Aug 10 01:02:45 dist-test-slave-gp6n krb5kdc[9013](info): commencing operation
krb5kdc: starting...
W20260810 01:02:46.193050 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.006s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:02:46 dist-test-slave-gp6n krb5kdc[9013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:02:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:02:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 01:02:46.522203 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:42751
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:42751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:02:46.872619 9028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:46.872936 9028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:46.873111 9028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:46.880456 9028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:02:46.880574 9028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:46.880666 9028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:46.880755 9028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:02:46.880872 9028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:02:46.890971 9028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:42751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:42751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:46.892354 9028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:46.893714 9028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:46.906064 9033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:46.905989 9034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:46.906045 9036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:46.907658 9028 server_base.cc:1034] running on GCE node
I20260810 01:02:46.908460 9028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:46.909896 9028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:46.911162 9028 hybrid_clock.cc:648] HybridClock initialized: now 1786323766911115 us; error 56 us; skew 500 ppm
I20260810 01:02:46.911568 9028 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:02:46 dist-test-slave-gp6n krb5kdc[9013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323766, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:02:46.921999 9028 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 01:02:46.923863 9028 webserver.cc:460] Webserver started at http://127.20.81.190:38119/ using document root <none> and password file <none>
I20260810 01:02:46.924664 9028 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:46.924844 9028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:46.925201 9028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:46.928869 9028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "650d5028fe914367a5033ba2d2290ca6"
format_stamp: "Formatted at 2026-08-10 01:02:46 on dist-test-slave-gp6n"
server_key: "25834d89037fff6c0f105abb3cd16d1f"
server_key_iv: "3f7031f422baafff1908de13e036028d"
server_key_version: "encryptionkey@0"
I20260810 01:02:46.929773 9028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "650d5028fe914367a5033ba2d2290ca6"
format_stamp: "Formatted at 2026-08-10 01:02:46 on dist-test-slave-gp6n"
server_key: "25834d89037fff6c0f105abb3cd16d1f"
server_key_iv: "3f7031f422baafff1908de13e036028d"
server_key_version: "encryptionkey@0"
I20260810 01:02:46.935599 9028 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260810 01:02:46.939561 9043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:46.940968 9028 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 01:02:46.941167 9028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "650d5028fe914367a5033ba2d2290ca6"
format_stamp: "Formatted at 2026-08-10 01:02:46 on dist-test-slave-gp6n"
server_key: "25834d89037fff6c0f105abb3cd16d1f"
server_key_iv: "3f7031f422baafff1908de13e036028d"
server_key_version: "encryptionkey@0"
I20260810 01:02:46.941372 9028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:46.983430 9028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:46.987896 9028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:46.988194 9028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:47.008750 9028 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:42751
I20260810 01:02:47.008785 9099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:42751 every 8 connection(s)
I20260810 01:02:47.010360 9028 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:02:47.014460 9100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:02:47.015350 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 9028
I20260810 01:02:47.015573 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:02:47.016222 20806 external_mini_cluster.cc:1428] Setting key 0fa967a32955d546253a709116fb4735
I20260810 01:02:47.029430 9100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6: Bootstrap starting.
Aug 10 01:02:47 dist-test-slave-gp6n krb5kdc[9013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:02:47.033452 9100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6: Neither blocks nor log segments found. Creating new log.
I20260810 01:02:47.034660 9100 log.cc:824] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:47.037734 9100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6: No bootstrap required, opened a new log
I20260810 01:02:47.041576 9103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:47.018358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50124 (local address 127.20.81.190:42751)
0810 01:02:47.019364 (+ 1006us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:47.019377 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:47.019424 (+ 47us) server_negotiation.cc:407] Connection header received
0810 01:02:47.021292 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:47.021328 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:47.021761 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:47.022122 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:47.024058 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:47.025441 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:47.027600 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:47.028305 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:47.028494 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:47.033696 (+ 5202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:47.033705 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:47.033719 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:47.033764 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:47.037173 (+ 3409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:47.037994 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:47.038001 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:47.038007 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:47.038181 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:47.038641 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:47.038646 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:47.038650 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:47.039197 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:47.039573 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:47.040027 (+ 454us) server_negotiation.cc:299] Negotiation successful
0810 01:02:47.040515 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":756,"thread_start_us":428,"threads_started":1}
I20260810 01:02:47.046011 9100 raft_consensus.cc:348] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "650d5028fe914367a5033ba2d2290ca6" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 42751 } }
I20260810 01:02:47.046391 9100 raft_consensus.cc:374] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:02:47.046555 9100 raft_consensus.cc:729] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 650d5028fe914367a5033ba2d2290ca6, State: Initialized, Role: FOLLOWER
I20260810 01:02:47.047183 9100 consensus_queue.cc:260] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "650d5028fe914367a5033ba2d2290ca6" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 42751 } }
I20260810 01:02:47.047503 9100 raft_consensus.cc:388] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:02:47.047664 9100 raft_consensus.cc:482] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:02:47.047854 9100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:02:47.050071 9100 raft_consensus.cc:504] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "650d5028fe914367a5033ba2d2290ca6" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 42751 } }
I20260810 01:02:47.050628 9100 leader_election.cc:302] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 650d5028fe914367a5033ba2d2290ca6; no voters:
I20260810 01:02:47.051222 9100 leader_election.cc:288] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:02:47.051414 9105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:02:47.052176 9105 raft_consensus.cc:686] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6 [term 1 LEADER]: Becoming Leader. State: Replica: 650d5028fe914367a5033ba2d2290ca6, State: Running, Role: LEADER
I20260810 01:02:47.052691 9105 consensus_queue.cc:237] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "650d5028fe914367a5033ba2d2290ca6" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 42751 } }
I20260810 01:02:47.053398 9100 sys_catalog.cc:564] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:02:47.055555 9106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "650d5028fe914367a5033ba2d2290ca6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "650d5028fe914367a5033ba2d2290ca6" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 42751 } } }
I20260810 01:02:47.056056 9106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6 [sys.catalog]: This master's current role is: LEADER
I20260810 01:02:47.055594 9107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 650d5028fe914367a5033ba2d2290ca6. Latest consensus state: current_term: 1 leader_uuid: "650d5028fe914367a5033ba2d2290ca6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "650d5028fe914367a5033ba2d2290ca6" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 42751 } } }
I20260810 01:02:47.056394 9107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6 [sys.catalog]: This master's current role is: LEADER
I20260810 01:02:47.057126 9112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:02:47.062870 9112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:02:47.071575 9112 catalog_manager.cc:1347] Generated new cluster ID: 974046039a1042568e410142dae824d6
I20260810 01:02:47.071691 9112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:02:47.081403 9112 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:02:47.082849 9112 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:02:47.098239 9112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6: Generated new TSK 0
I20260810 01:02:47.099318 9112 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.81.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
I20260810 01:02:47.186477 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:42751
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:02:47.546322 9127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:47.546666 9127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:47.546764 9127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:47.554067 9127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:47.554163 9127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:47.554330 9127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 01:02:47.564232 9127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:42751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:47.565421 9127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:47.566717 9127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:47.580955 9136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:47.580901 9134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:47.581001 9133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:47.582151 9127 server_base.cc:1034] running on GCE node
I20260810 01:02:47.582794 9127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:47.583724 9127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:47.584990 9127 hybrid_clock.cc:648] HybridClock initialized: now 1786323767584938 us; error 42 us; skew 500 ppm
I20260810 01:02:47.585326 9127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:02:47 dist-test-slave-gp6n krb5kdc[9013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323767, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:02:47.588666 9127 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 01:02:47.590148 9127 webserver.cc:460] Webserver started at http://127.20.81.129:43659/ using document root <none> and password file <none>
I20260810 01:02:47.590850 9127 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:47.591017 9127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:47.591434 9127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:47.594491 9127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "2d96e48a28ed43a593d77bbc96957590"
format_stamp: "Formatted at 2026-08-10 01:02:47 on dist-test-slave-gp6n"
server_key: "d2e350077f28b4d6d258758b16da65c1"
server_key_iv: "2e894aabb107bd371d2c552edd2bce9a"
server_key_version: "encryptionkey@0"
I20260810 01:02:47.595300 9127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "2d96e48a28ed43a593d77bbc96957590"
format_stamp: "Formatted at 2026-08-10 01:02:47 on dist-test-slave-gp6n"
server_key: "d2e350077f28b4d6d258758b16da65c1"
server_key_iv: "2e894aabb107bd371d2c552edd2bce9a"
server_key_version: "encryptionkey@0"
I20260810 01:02:47.600775 9127 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 01:02:47.604283 9144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:47.605672 9127 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 01:02:47.605937 9127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "2d96e48a28ed43a593d77bbc96957590"
format_stamp: "Formatted at 2026-08-10 01:02:47 on dist-test-slave-gp6n"
server_key: "d2e350077f28b4d6d258758b16da65c1"
server_key_iv: "2e894aabb107bd371d2c552edd2bce9a"
server_key_version: "encryptionkey@0"
I20260810 01:02:47.606149 9127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:47.625658 9127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:47.630025 9127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:47.630308 9127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:47.631295 9127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:02:47.632566 9127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:02:47.632638 9127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:47.632723 9127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:02:47.632766 9127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:47.668046 9127 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:35635
I20260810 01:02:47.668138 9281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:35635 every 8 connection(s)
I20260810 01:02:47.669451 9127 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 01:02:47.670480 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 9127
I20260810 01:02:47.670670 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 01:02:47.671188 20806 external_mini_cluster.cc:1428] Setting key f8c97a2d55029efcf8725fa13cf04feb
Aug 10 01:02:47 dist-test-slave-gp6n krb5kdc[9013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323767, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:02:47.691310 9284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:47.672465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:44567 (local address 127.20.81.190:42751)
0810 01:02:47.672963 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:47.672970 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:47.674097 (+ 1127us) server_negotiation.cc:407] Connection header received
0810 01:02:47.676232 (+ 2135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:47.676237 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:47.676337 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:47.676464 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:47.678107 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:47.678996 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:47.680709 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:47.681157 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:47.681316 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:47.685168 (+ 3852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:47.685173 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:47.685177 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:47.685210 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:47.687988 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:47.688951 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:47.688955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:47.688958 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:47.689084 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:47.689596 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:47.689600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:47.689602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:47.689866 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:47.689980 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:47.690887 (+ 907us) server_negotiation.cc:299] Negotiation successful
0810 01:02:47.691056 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":244,"threads_started":1}
I20260810 01:02:47.693044 9286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:47.673088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:42751 (local address 127.20.81.129:44567)
0810 01:02:47.673919 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:47.673969 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:47.675934 (+ 1965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:47.676741 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:47.676750 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:47.677318 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:02:47.677923 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:47.677942 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:47.679159 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:47.679165 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:47.680545 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:47.680558 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:47.681425 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:47.681433 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:47.681694 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:47.682510 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:02:47.682541 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:02:47.684930 (+ 2389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:02:47.688153 (+ 3223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:02:47.688161 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:02:47.688187 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:02:47.688822 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:02:47.689200 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:02:47.689207 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:02:47.689214 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:02:47.689482 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:02:47.689993 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:02:47.690003 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:02:47.690487 (+ 484us) client_negotiation.cc:769] Sending connection context
0810 01:02:47.690814 (+ 327us) client_negotiation.cc:240] Negotiation successful
0810 01:02:47.691542 (+ 728us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":342,"threads_started":1}
I20260810 01:02:47.694764 9282 heartbeater.cc:347] Connected to a master server at 127.20.81.190:42751
I20260810 01:02:47.695094 9282 heartbeater.cc:464] Registering TS with master...
I20260810 01:02:47.695994 9282 heartbeater.cc:511] Master 127.20.81.190:42751 requested a full tablet report, sending...
I20260810 01:02:47.698292 9064 ts_manager.cc:194] Registered new tserver with Master: 2d96e48a28ed43a593d77bbc96957590 (127.20.81.129:35635)
I20260810 01:02:47.699796 9064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:44567
WARNING: no policy specified for kudu/127.20.81.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
I20260810 01:02:47.748303 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:42751
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:02:48.109427 9289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:48.109737 9289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:48.109910 9289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:48.117208 9289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:48.117383 9289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:48.117606 9289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 01:02:48.127591 9289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:42751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:48.128906 9289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:48.130271 9289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:48.143586 9295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:48.143659 9299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:48.143663 9296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:48.144884 9289 server_base.cc:1034] running on GCE node
I20260810 01:02:48.145499 9289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:48.146572 9289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:48.147828 9289 hybrid_clock.cc:648] HybridClock initialized: now 1786323768147777 us; error 52 us; skew 500 ppm
I20260810 01:02:48.148223 9289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:02:48 dist-test-slave-gp6n krb5kdc[9013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323768, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:02:48.151984 9289 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 01:02:48.153651 9289 webserver.cc:460] Webserver started at http://127.20.81.130:40685/ using document root <none> and password file <none>
I20260810 01:02:48.154518 9289 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:48.154678 9289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:48.155040 9289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:48.158280 9289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "60a8395609fe497b89946b00039d67f2"
format_stamp: "Formatted at 2026-08-10 01:02:48 on dist-test-slave-gp6n"
server_key: "36dfbad5dd2a019126a1be65ec0581b8"
server_key_iv: "722f2e95d971e3b551501a3c2e5f90dd"
server_key_version: "encryptionkey@0"
I20260810 01:02:48.159212 9289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "60a8395609fe497b89946b00039d67f2"
format_stamp: "Formatted at 2026-08-10 01:02:48 on dist-test-slave-gp6n"
server_key: "36dfbad5dd2a019126a1be65ec0581b8"
server_key_iv: "722f2e95d971e3b551501a3c2e5f90dd"
server_key_version: "encryptionkey@0"
I20260810 01:02:48.164683 9289 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 01:02:48.168535 9306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:48.169937 9289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 01:02:48.170130 9289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "60a8395609fe497b89946b00039d67f2"
format_stamp: "Formatted at 2026-08-10 01:02:48 on dist-test-slave-gp6n"
server_key: "36dfbad5dd2a019126a1be65ec0581b8"
server_key_iv: "722f2e95d971e3b551501a3c2e5f90dd"
server_key_version: "encryptionkey@0"
I20260810 01:02:48.170343 9289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:48.185474 9289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:48.190233 9289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:48.190587 9289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:48.191713 9289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:02:48.193143 9289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:02:48.193224 9289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:48.193318 9289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:02:48.193373 9289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:48.237721 9289 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:33269
I20260810 01:02:48.237766 9432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:33269 every 8 connection(s)
I20260810 01:02:48.239295 9289 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 01:02:48.240528 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 9289
I20260810 01:02:48.240743 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 01:02:48.241281 20806 external_mini_cluster.cc:1428] Setting key 1cf590fff7002bbb0c8b944fc62fab92
Aug 10 01:02:48 dist-test-slave-gp6n krb5kdc[9013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323768, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:02:48.263051 9435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:48.242678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:34105 (local address 127.20.81.190:42751)
0810 01:02:48.243211 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:48.243220 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:48.243862 (+ 642us) server_negotiation.cc:407] Connection header received
0810 01:02:48.246111 (+ 2249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:48.246120 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:48.246217 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:48.246343 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:48.247953 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:48.249249 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:48.250990 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:48.251811 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:48.251975 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:48.255697 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:48.255707 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:48.255716 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:48.255768 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:48.259607 (+ 3839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:48.260560 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:48.260564 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:48.260566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:48.260705 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:48.261237 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:48.261240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:48.261243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:48.261513 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:48.261642 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:48.262594 (+ 952us) server_negotiation.cc:299] Negotiation successful
0810 01:02:48.262772 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":264,"threads_started":1}
I20260810 01:02:48.264863 9436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:48.242898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:42751 (local address 127.20.81.130:34105)
0810 01:02:48.243697 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:48.243740 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:48.245805 (+ 2065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:48.246657 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:48.246666 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:48.247181 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:02:48.247764 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:48.247783 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:48.249459 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:48.249463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:48.250841 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:48.250853 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:48.252054 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:48.252060 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:48.252258 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:48.253037 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:02:48.253061 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:02:48.255439 (+ 2378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:02:48.259769 (+ 4330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:02:48.259779 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:02:48.259797 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:02:48.260405 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:02:48.260829 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:02:48.260837 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:02:48.260844 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:02:48.261116 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:02:48.261645 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:02:48.261656 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:02:48.262147 (+ 491us) client_negotiation.cc:769] Sending connection context
0810 01:02:48.262510 (+ 363us) client_negotiation.cc:240] Negotiation successful
0810 01:02:48.263177 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":934,"threads_started":1}
I20260810 01:02:48.266394 9433 heartbeater.cc:347] Connected to a master server at 127.20.81.190:42751
I20260810 01:02:48.266754 9433 heartbeater.cc:464] Registering TS with master...
I20260810 01:02:48.267505 9433 heartbeater.cc:511] Master 127.20.81.190:42751 requested a full tablet report, sending...
I20260810 01:02:48.268973 9064 ts_manager.cc:194] Registered new tserver with Master: 60a8395609fe497b89946b00039d67f2 (127.20.81.130:33269)
I20260810 01:02:48.269806 9064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:34105
WARNING: no policy specified for kudu/127.20.81.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
I20260810 01:02:48.318692 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:42751
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:44511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:02:48.668954 9439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:48.669206 9439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:48.669378 9439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:48.677438 9439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:48.677538 9439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:48.677754 9439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 01:02:48.687479 9439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:44511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:42751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:48.688684 9439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:48.690012 9439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 01:02:48.703714 9282 heartbeater.cc:503] Master 127.20.81.190:42751 was elected leader, sending a full tablet report...
W20260810 01:02:48.706286 9445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:48.706483 9448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:48.706569 9444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:48.707988 9439 server_base.cc:1034] running on GCE node
I20260810 01:02:48.709542 9439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:48.710958 9439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:48.712218 9439 hybrid_clock.cc:648] HybridClock initialized: now 1786323768712172 us; error 36 us; skew 500 ppm
I20260810 01:02:48.712551 9439 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:02:48 dist-test-slave-gp6n krb5kdc[9013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323768, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:02:48.715732 9439 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 01:02:48.717201 9439 webserver.cc:460] Webserver started at http://127.20.81.131:37783/ using document root <none> and password file <none>
I20260810 01:02:48.717900 9439 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:48.718048 9439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:48.718379 9439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:48.721428 9439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "5b000243ac8c40709252a1381aff4ff4"
format_stamp: "Formatted at 2026-08-10 01:02:48 on dist-test-slave-gp6n"
server_key: "87011d6accab45318db8758d68cb62a3"
server_key_iv: "359af6c6fda0637eeb40510bc5258bb8"
server_key_version: "encryptionkey@0"
I20260810 01:02:48.722270 9439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "5b000243ac8c40709252a1381aff4ff4"
format_stamp: "Formatted at 2026-08-10 01:02:48 on dist-test-slave-gp6n"
server_key: "87011d6accab45318db8758d68cb62a3"
server_key_iv: "359af6c6fda0637eeb40510bc5258bb8"
server_key_version: "encryptionkey@0"
I20260810 01:02:48.728266 9439 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260810 01:02:48.732017 9457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:48.733356 9439 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 01:02:48.733536 9439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "5b000243ac8c40709252a1381aff4ff4"
format_stamp: "Formatted at 2026-08-10 01:02:48 on dist-test-slave-gp6n"
server_key: "87011d6accab45318db8758d68cb62a3"
server_key_iv: "359af6c6fda0637eeb40510bc5258bb8"
server_key_version: "encryptionkey@0"
I20260810 01:02:48.733758 9439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:48.748255 9439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:48.752336 9439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:48.752640 9439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:48.753700 9439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:02:48.755187 9439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:02:48.755261 9439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:48.755406 9439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:02:48.755466 9439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:48.790807 9439 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:42871
I20260810 01:02:48.790889 9586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:42871 every 8 connection(s)
I20260810 01:02:48.792178 9439 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 01:02:48.793409 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 9439
I20260810 01:02:48.793615 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 01:02:48.794219 20806 external_mini_cluster.cc:1428] Setting key ad2b3740e6816f1ba7925fa742e14889
Aug 10 01:02:48 dist-test-slave-gp6n krb5kdc[9013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323768, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:02:48.813475 9589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:48.795108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:53959 (local address 127.20.81.190:42751)
0810 01:02:48.795927 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:48.795934 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:48.796802 (+ 868us) server_negotiation.cc:407] Connection header received
0810 01:02:48.799033 (+ 2231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:48.799038 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:48.799133 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:48.799246 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:48.800772 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:48.801553 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:48.803205 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:48.803617 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:48.803749 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:48.807287 (+ 3538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:48.807292 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:48.807295 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:48.807329 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:48.810811 (+ 3482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:48.811507 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:48.811511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:48.811514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:48.811634 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:48.812016 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:48.812019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:48.812021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:48.812274 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:48.812391 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:48.813022 (+ 631us) server_negotiation.cc:299] Negotiation successful
0810 01:02:48.813183 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":566,"threads_started":1}
I20260810 01:02:48.814445 9591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:48.795748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:42751 (local address 127.20.81.131:53959)
0810 01:02:48.796627 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:48.796692 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:48.798702 (+ 2010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:48.799505 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:48.799513 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:48.800031 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:02:48.800599 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:48.800617 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:48.801709 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:48.801714 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:48.803049 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:48.803061 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:48.803865 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:48.803873 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:48.804101 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:48.804782 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:02:48.804805 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:02:48.807076 (+ 2271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:02:48.810955 (+ 3879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:02:48.810962 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:02:48.810975 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:02:48.811388 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:02:48.811724 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:02:48.811726 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:02:48.811728 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:02:48.811915 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:02:48.812400 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:02:48.812405 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:02:48.812702 (+ 297us) client_negotiation.cc:769] Sending connection context
0810 01:02:48.812922 (+ 220us) client_negotiation.cc:240] Negotiation successful
0810 01:02:48.813292 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":331,"threads_started":1}
I20260810 01:02:48.815702 9587 heartbeater.cc:347] Connected to a master server at 127.20.81.190:42751
I20260810 01:02:48.816015 9587 heartbeater.cc:464] Registering TS with master...
I20260810 01:02:48.816749 9587 heartbeater.cc:511] Master 127.20.81.190:42751 requested a full tablet report, sending...
I20260810 01:02:48.818236 9064 ts_manager.cc:194] Registered new tserver with Master: 5b000243ac8c40709252a1381aff4ff4 (127.20.81.131:42871)
I20260810 01:02:48.819171 9064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:53959
I20260810 01:02:48.822587 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 01:02:48.839603 9589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:48.826698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50140 (local address 127.20.81.190:42751)
0810 01:02:48.826861 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:48.826866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:48.827293 (+ 427us) server_negotiation.cc:407] Connection header received
0810 01:02:48.827467 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:48.827470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:48.827547 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:48.827625 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:48.828958 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:48.829675 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:48.831094 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:48.831476 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:48.831576 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:48.834264 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:48.834269 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:48.834273 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:48.834301 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:48.836763 (+ 2462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:48.837645 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:48.837649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:48.837651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:48.837769 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:48.838173 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:48.838176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:48.838178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:48.838551 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:48.838790 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:48.839101 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 01:02:48.839236 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 01:02:48.843705 9064 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50140:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 01:02:48.850871 9064 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 01:02:48.876497 9602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:48.863157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:35635 (local address 127.0.0.1:38210)
0810 01:02:48.865337 (+ 2180us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:48.865380 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:48.865609 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:48.866798 (+ 1189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:48.866806 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:48.866839 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:48.867281 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:48.867293 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:48.869869 (+ 2576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:48.869879 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:48.872767 (+ 2888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:48.872789 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:48.875471 (+ 2682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:48.875489 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:48.875731 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:48.875803 (+ 72us) client_negotiation.cc:769] Sending connection context
0810 01:02:48.875936 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 01:02:48.876129 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2002,"thread_start_us":1871,"threads_started":1}
I20260810 01:02:48.877589 9603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:48.863315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38210 (local address 127.20.81.129:35635)
0810 01:02:48.865052 (+ 1737us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:48.865064 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:48.865505 (+ 441us) server_negotiation.cc:407] Connection header received
0810 01:02:48.865925 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:48.865938 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:48.866256 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:48.866506 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:48.867542 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:48.869651 (+ 2109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:48.873013 (+ 3362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:48.874943 (+ 1930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:48.875079 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:48.875344 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:48.877016 (+ 1672us) server_negotiation.cc:299] Negotiation successful
0810 01:02:48.877219 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1539,"thread_start_us":1378,"threads_started":1}
I20260810 01:02:48.882375 9203 tablet_service.cc:1467] Processing CreateTablet for tablet 547df716a8954d53b0f717f6a68c13e0 (DEFAULT_TABLE table=test-table [id=97ce7a0779fd4e3dacca9abe27d4407d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:02:48.883446 9605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:48.863968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:33269 (local address 127.0.0.1:45170)
0810 01:02:48.875534 (+ 11566us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:48.875570 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:48.875719 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:48.876933 (+ 1214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:48.876941 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:48.876965 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:48.877330 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:48.877343 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:48.879372 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:48.879377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:48.881242 (+ 1865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:48.881253 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:48.882542 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:48.882548 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:48.882795 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:48.882818 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 01:02:48.882929 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 01:02:48.883061 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10777,"mutex_wait_us":42,"thread_start_us":11503,"threads_started":1}
I20260810 01:02:48.883932 9203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 547df716a8954d53b0f717f6a68c13e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:02:48.884276 9606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:48.864077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45170 (local address 127.20.81.130:33269)
0810 01:02:48.876061 (+ 11984us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:48.876070 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:48.876099 (+ 29us) server_negotiation.cc:407] Connection header received
0810 01:02:48.876195 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:48.876203 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:48.876473 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:48.876700 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:48.877497 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:48.879229 (+ 1732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:48.881394 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:48.882296 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:48.882476 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:48.882685 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:48.883808 (+ 1123us) server_negotiation.cc:299] Negotiation successful
0810 01:02:48.883988 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11802,"thread_start_us":11741,"threads_started":1}
I20260810 01:02:48.885629 9602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:48.866034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:42871 (local address 127.0.0.1:51368)
0810 01:02:48.876753 (+ 10719us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:48.876776 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:48.876861 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:48.878654 (+ 1793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:48.878660 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:48.878689 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:48.878882 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:48.878892 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:48.880870 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:48.880877 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:48.882768 (+ 1891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:48.882799 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:48.884829 (+ 2030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:48.884834 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:48.885028 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:48.885064 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 01:02:48.885160 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 01:02:48.885305 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10633,"mutex_wait_us":39,"thread_start_us":11964,"threads_started":1}
I20260810 01:02:48.886849 9608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:48.866735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51368 (local address 127.20.81.131:42871)
0810 01:02:48.878049 (+ 11314us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:48.878060 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:48.878095 (+ 35us) server_negotiation.cc:407] Connection header received
0810 01:02:48.878195 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:48.878209 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:48.878447 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:48.878646 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:48.879050 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:48.880702 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:48.882955 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:48.884395 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:48.884523 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:48.884736 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:48.885239 (+ 503us) server_negotiation.cc:299] Negotiation successful
0810 01:02:48.885392 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11093,"thread_start_us":10899,"threads_started":1}
I20260810 01:02:48.887578 9362 tablet_service.cc:1467] Processing CreateTablet for tablet 547df716a8954d53b0f717f6a68c13e0 (DEFAULT_TABLE table=test-table [id=97ce7a0779fd4e3dacca9abe27d4407d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:02:48.888636 9362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 547df716a8954d53b0f717f6a68c13e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:02:48.893940 9513 tablet_service.cc:1467] Processing CreateTablet for tablet 547df716a8954d53b0f717f6a68c13e0 (DEFAULT_TABLE table=test-table [id=97ce7a0779fd4e3dacca9abe27d4407d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:02:48.895430 9513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 547df716a8954d53b0f717f6a68c13e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:02:48.897053 9610 tablet_bootstrap.cc:492] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590: Bootstrap starting.
I20260810 01:02:48.900810 9610 tablet_bootstrap.cc:654] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590: Neither blocks nor log segments found. Creating new log.
I20260810 01:02:48.901041 9611 tablet_bootstrap.cc:492] T 547df716a8954d53b0f717f6a68c13e0 P 60a8395609fe497b89946b00039d67f2: Bootstrap starting.
I20260810 01:02:48.902872 9610 log.cc:824] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:48.904544 9611 tablet_bootstrap.cc:654] T 547df716a8954d53b0f717f6a68c13e0 P 60a8395609fe497b89946b00039d67f2: Neither blocks nor log segments found. Creating new log.
I20260810 01:02:48.905613 9610 tablet_bootstrap.cc:492] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590: No bootstrap required, opened a new log
I20260810 01:02:48.905813 9611 log.cc:824] T 547df716a8954d53b0f717f6a68c13e0 P 60a8395609fe497b89946b00039d67f2: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:48.905880 9610 ts_tablet_manager.cc:1397] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260810 01:02:48.908916 9611 tablet_bootstrap.cc:492] T 547df716a8954d53b0f717f6a68c13e0 P 60a8395609fe497b89946b00039d67f2: No bootstrap required, opened a new log
I20260810 01:02:48.909240 9611 ts_tablet_manager.cc:1397] T 547df716a8954d53b0f717f6a68c13e0 P 60a8395609fe497b89946b00039d67f2: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260810 01:02:48.909871 9614 tablet_bootstrap.cc:492] T 547df716a8954d53b0f717f6a68c13e0 P 5b000243ac8c40709252a1381aff4ff4: Bootstrap starting.
I20260810 01:02:48.914186 9614 tablet_bootstrap.cc:654] T 547df716a8954d53b0f717f6a68c13e0 P 5b000243ac8c40709252a1381aff4ff4: Neither blocks nor log segments found. Creating new log.
I20260810 01:02:48.915380 9614 log.cc:824] T 547df716a8954d53b0f717f6a68c13e0 P 5b000243ac8c40709252a1381aff4ff4: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:48.915499 9610 raft_consensus.cc:348] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d96e48a28ed43a593d77bbc96957590" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 35635 } } peers { permanent_uuid: "5b000243ac8c40709252a1381aff4ff4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42871 } } peers { permanent_uuid: "60a8395609fe497b89946b00039d67f2" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 33269 } }
I20260810 01:02:48.915951 9610 raft_consensus.cc:374] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:02:48.916079 9610 raft_consensus.cc:729] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d96e48a28ed43a593d77bbc96957590, State: Initialized, Role: FOLLOWER
I20260810 01:02:48.916715 9610 consensus_queue.cc:260] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d96e48a28ed43a593d77bbc96957590" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 35635 } } peers { permanent_uuid: "5b000243ac8c40709252a1381aff4ff4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42871 } } peers { permanent_uuid: "60a8395609fe497b89946b00039d67f2" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 33269 } }
I20260810 01:02:48.917797 9611 raft_consensus.cc:348] T 547df716a8954d53b0f717f6a68c13e0 P 60a8395609fe497b89946b00039d67f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d96e48a28ed43a593d77bbc96957590" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 35635 } } peers { permanent_uuid: "5b000243ac8c40709252a1381aff4ff4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42871 } } peers { permanent_uuid: "60a8395609fe497b89946b00039d67f2" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 33269 } }
I20260810 01:02:48.918220 9611 raft_consensus.cc:374] T 547df716a8954d53b0f717f6a68c13e0 P 60a8395609fe497b89946b00039d67f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:02:48.918308 9611 raft_consensus.cc:729] T 547df716a8954d53b0f717f6a68c13e0 P 60a8395609fe497b89946b00039d67f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60a8395609fe497b89946b00039d67f2, State: Initialized, Role: FOLLOWER
I20260810 01:02:48.918448 9610 ts_tablet_manager.cc:1428] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590: Time spent starting tablet: real 0.012s user 0.005s sys 0.009s
I20260810 01:02:48.918746 9614 tablet_bootstrap.cc:492] T 547df716a8954d53b0f717f6a68c13e0 P 5b000243ac8c40709252a1381aff4ff4: No bootstrap required, opened a new log
I20260810 01:02:48.918927 9614 ts_tablet_manager.cc:1397] T 547df716a8954d53b0f717f6a68c13e0 P 5b000243ac8c40709252a1381aff4ff4: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260810 01:02:48.919023 9611 consensus_queue.cc:260] T 547df716a8954d53b0f717f6a68c13e0 P 60a8395609fe497b89946b00039d67f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d96e48a28ed43a593d77bbc96957590" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 35635 } } peers { permanent_uuid: "5b000243ac8c40709252a1381aff4ff4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42871 } } peers { permanent_uuid: "60a8395609fe497b89946b00039d67f2" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 33269 } }
I20260810 01:02:48.920252 9433 heartbeater.cc:503] Master 127.20.81.190:42751 was elected leader, sending a full tablet report...
I20260810 01:02:48.920817 9611 ts_tablet_manager.cc:1428] T 547df716a8954d53b0f717f6a68c13e0 P 60a8395609fe497b89946b00039d67f2: Time spent starting tablet: real 0.011s user 0.007s sys 0.005s
W20260810 01:02:48.922019 9283 tablet.cc:2369] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:02:48.929920 9614 raft_consensus.cc:348] T 547df716a8954d53b0f717f6a68c13e0 P 5b000243ac8c40709252a1381aff4ff4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d96e48a28ed43a593d77bbc96957590" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 35635 } } peers { permanent_uuid: "5b000243ac8c40709252a1381aff4ff4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42871 } } peers { permanent_uuid: "60a8395609fe497b89946b00039d67f2" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 33269 } }
I20260810 01:02:48.930459 9614 raft_consensus.cc:374] T 547df716a8954d53b0f717f6a68c13e0 P 5b000243ac8c40709252a1381aff4ff4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:02:48.930594 9614 raft_consensus.cc:729] T 547df716a8954d53b0f717f6a68c13e0 P 5b000243ac8c40709252a1381aff4ff4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b000243ac8c40709252a1381aff4ff4, State: Initialized, Role: FOLLOWER
I20260810 01:02:48.931252 9614 consensus_queue.cc:260] T 547df716a8954d53b0f717f6a68c13e0 P 5b000243ac8c40709252a1381aff4ff4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d96e48a28ed43a593d77bbc96957590" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 35635 } } peers { permanent_uuid: "5b000243ac8c40709252a1381aff4ff4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42871 } } peers { permanent_uuid: "60a8395609fe497b89946b00039d67f2" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 33269 } }
I20260810 01:02:48.932399 9587 heartbeater.cc:503] Master 127.20.81.190:42751 was elected leader, sending a full tablet report...
I20260810 01:02:48.933050 9614 ts_tablet_manager.cc:1428] T 547df716a8954d53b0f717f6a68c13e0 P 5b000243ac8c40709252a1381aff4ff4: Time spent starting tablet: real 0.014s user 0.006s sys 0.005s
W20260810 01:02:48.991468 9434 tablet.cc:2369] T 547df716a8954d53b0f717f6a68c13e0 P 60a8395609fe497b89946b00039d67f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 01:02:49.043889 9588 tablet.cc:2369] T 547df716a8954d53b0f717f6a68c13e0 P 5b000243ac8c40709252a1381aff4ff4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:02:49.166725 9616 raft_consensus.cc:482] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:02:49.167128 9616 raft_consensus.cc:504] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d96e48a28ed43a593d77bbc96957590" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 35635 } } peers { permanent_uuid: "5b000243ac8c40709252a1381aff4ff4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42871 } } peers { permanent_uuid: "60a8395609fe497b89946b00039d67f2" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 33269 } }
I20260810 01:02:49.169137 9616 leader_election.cc:288] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b000243ac8c40709252a1381aff4ff4 (127.20.81.131:42871), 60a8395609fe497b89946b00039d67f2 (127.20.81.130:33269)
I20260810 01:02:49.177114 9606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:49.169943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:49097 (local address 127.20.81.130:33269)
0810 01:02:49.170213 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:49.170222 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:49.170682 (+ 460us) server_negotiation.cc:407] Connection header received
0810 01:02:49.170994 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:49.170999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:49.171062 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:49.171177 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:49.171972 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:49.172718 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:49.175014 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:49.176028 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:49.176157 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:49.176243 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:49.176645 (+ 402us) server_negotiation.cc:299] Negotiation successful
0810 01:02:49.176799 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 01:02:49.177120 9608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:49.169564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:55927 (local address 127.20.81.131:42871)
0810 01:02:49.169842 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:49.169853 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:49.170764 (+ 911us) server_negotiation.cc:407] Connection header received
0810 01:02:49.170950 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:49.170956 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:49.171049 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:49.171127 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:49.171827 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:49.172718 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:49.175194 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:49.176142 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:49.176325 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:49.176415 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:49.176731 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 01:02:49.176876 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 01:02:49.177205 9619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:49.169570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:33269 (local address 127.20.81.129:49097)
0810 01:02:49.170561 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:49.170579 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:49.170784 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:49.171225 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:49.171230 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:49.171257 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:49.171812 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:49.171824 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:49.172888 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:49.172894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:49.174807 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:49.174820 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:49.176256 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:49.176263 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:49.176458 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:49.176496 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 01:02:49.176576 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 01:02:49.176705 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"mutex_wait_us":44,"thread_start_us":666,"threads_started":1}
I20260810 01:02:49.177793 9620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:49.169590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:42871 (local address 127.20.81.129:55927)
0810 01:02:49.170666 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:49.170682 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:49.170803 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:49.171177 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:49.171182 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:49.171208 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:49.171656 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:49.171668 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:49.172881 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:49.172890 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:49.174988 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:49.175004 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:49.176390 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:49.176395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:49.176532 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:49.176587 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 01:02:49.176692 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 01:02:49.176827 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":928,"mutex_wait_us":49,"thread_start_us":662,"threads_started":1}
I20260810 01:02:49.178387 9384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "547df716a8954d53b0f717f6a68c13e0" candidate_uuid: "2d96e48a28ed43a593d77bbc96957590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60a8395609fe497b89946b00039d67f2" is_pre_election: true
I20260810 01:02:49.178889 9535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "547df716a8954d53b0f717f6a68c13e0" candidate_uuid: "2d96e48a28ed43a593d77bbc96957590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b000243ac8c40709252a1381aff4ff4" is_pre_election: true
I20260810 01:02:49.179069 9384 raft_consensus.cc:2454] T 547df716a8954d53b0f717f6a68c13e0 P 60a8395609fe497b89946b00039d67f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d96e48a28ed43a593d77bbc96957590 in term 0.
I20260810 01:02:49.179433 9535 raft_consensus.cc:2454] T 547df716a8954d53b0f717f6a68c13e0 P 5b000243ac8c40709252a1381aff4ff4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d96e48a28ed43a593d77bbc96957590 in term 0.
I20260810 01:02:49.179831 9145 leader_election.cc:302] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d96e48a28ed43a593d77bbc96957590, 60a8395609fe497b89946b00039d67f2; no voters:
I20260810 01:02:49.180289 9616 raft_consensus.cc:2781] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:02:49.180445 9616 raft_consensus.cc:482] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:02:49.180572 9616 raft_consensus.cc:3037] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:02:49.182937 9616 raft_consensus.cc:504] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d96e48a28ed43a593d77bbc96957590" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 35635 } } peers { permanent_uuid: "5b000243ac8c40709252a1381aff4ff4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42871 } } peers { permanent_uuid: "60a8395609fe497b89946b00039d67f2" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 33269 } }
I20260810 01:02:49.183662 9616 leader_election.cc:288] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590 [CANDIDATE]: Term 1 election: Requested vote from peers 5b000243ac8c40709252a1381aff4ff4 (127.20.81.131:42871), 60a8395609fe497b89946b00039d67f2 (127.20.81.130:33269)
I20260810 01:02:49.184024 9535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "547df716a8954d53b0f717f6a68c13e0" candidate_uuid: "2d96e48a28ed43a593d77bbc96957590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b000243ac8c40709252a1381aff4ff4"
I20260810 01:02:49.184231 9535 raft_consensus.cc:3037] T 547df716a8954d53b0f717f6a68c13e0 P 5b000243ac8c40709252a1381aff4ff4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:02:49.184206 9384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "547df716a8954d53b0f717f6a68c13e0" candidate_uuid: "2d96e48a28ed43a593d77bbc96957590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60a8395609fe497b89946b00039d67f2"
I20260810 01:02:49.184618 9384 raft_consensus.cc:3037] T 547df716a8954d53b0f717f6a68c13e0 P 60a8395609fe497b89946b00039d67f2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:02:49.186532 9535 raft_consensus.cc:2454] T 547df716a8954d53b0f717f6a68c13e0 P 5b000243ac8c40709252a1381aff4ff4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d96e48a28ed43a593d77bbc96957590 in term 1.
I20260810 01:02:49.186955 9148 leader_election.cc:302] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d96e48a28ed43a593d77bbc96957590, 5b000243ac8c40709252a1381aff4ff4; no voters:
I20260810 01:02:49.187247 9616 raft_consensus.cc:2781] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:02:49.187528 9384 raft_consensus.cc:2454] T 547df716a8954d53b0f717f6a68c13e0 P 60a8395609fe497b89946b00039d67f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d96e48a28ed43a593d77bbc96957590 in term 1.
I20260810 01:02:49.187776 9616 raft_consensus.cc:686] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590 [term 1 LEADER]: Becoming Leader. State: Replica: 2d96e48a28ed43a593d77bbc96957590, State: Running, Role: LEADER
I20260810 01:02:49.188306 9616 consensus_queue.cc:237] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d96e48a28ed43a593d77bbc96957590" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 35635 } } peers { permanent_uuid: "5b000243ac8c40709252a1381aff4ff4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42871 } } peers { permanent_uuid: "60a8395609fe497b89946b00039d67f2" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 33269 } }
I20260810 01:02:49.192857 9062 catalog_manager.cc:5495] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d96e48a28ed43a593d77bbc96957590 (127.20.81.129). New cstate: current_term: 1 leader_uuid: "2d96e48a28ed43a593d77bbc96957590" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d96e48a28ed43a593d77bbc96957590" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 35635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b000243ac8c40709252a1381aff4ff4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60a8395609fe497b89946b00039d67f2" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 33269 } health_report { overall_health: UNKNOWN } } }
I20260810 01:02:49.243129 9603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:49.237008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38218 (local address 127.20.81.129:35635)
0810 01:02:49.237229 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:49.237235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:49.237408 (+ 173us) server_negotiation.cc:407] Connection header received
0810 01:02:49.237658 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:49.237663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:49.237765 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:49.237933 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:02:49.238441 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:49.239189 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:49.240679 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:49.241109 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:49.241249 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:49.241898 (+ 649us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:02:49.242405 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:02:49.242616 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:49.242755 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 01:02:49.242797 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 01:02:49.251844 9535 raft_consensus.cc:1261] T 547df716a8954d53b0f717f6a68c13e0 P 5b000243ac8c40709252a1381aff4ff4 [term 1 FOLLOWER]: Refusing update from remote peer 2d96e48a28ed43a593d77bbc96957590: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:02:49.251844 9384 raft_consensus.cc:1261] T 547df716a8954d53b0f717f6a68c13e0 P 60a8395609fe497b89946b00039d67f2 [term 1 FOLLOWER]: Refusing update from remote peer 2d96e48a28ed43a593d77bbc96957590: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:02:49.252907 9621 consensus_queue.cc:1035] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b000243ac8c40709252a1381aff4ff4" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 42871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:02:49.253338 9616 consensus_queue.cc:1035] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60a8395609fe497b89946b00039d67f2" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:02:49.265699 9624 mvcc.cc:205] Tried to move back new op lower bound from 7316782158844854272 to 7316782158599274496. Current Snapshot: MvccSnapshot[applied={T|T < 7316782158844854272}]
I20260810 01:02:49.267590 9625 mvcc.cc:205] Tried to move back new op lower bound from 7316782158844854272 to 7316782158599274496. Current Snapshot: MvccSnapshot[applied={T|T < 7316782158844854272 or (T in {7316782158844854272})}]
I20260810 01:02:49.284334 9062 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:02:49.284636 9062 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:02:49.288367 9062 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 650d5028fe914367a5033ba2d2290ca6: Sending DeleteTablet for 3 replicas of tablet 547df716a8954d53b0f717f6a68c13e0
I20260810 01:02:49.289723 9362 tablet_service.cc:1514] Processing DeleteTablet for tablet 547df716a8954d53b0f717f6a68c13e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:02:49 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:45170
I20260810 01:02:49.289901 9513 tablet_service.cc:1514] Processing DeleteTablet for tablet 547df716a8954d53b0f717f6a68c13e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:02:49 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:51368
I20260810 01:02:49.290726 9203 tablet_service.cc:1514] Processing DeleteTablet for tablet 547df716a8954d53b0f717f6a68c13e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:02:49 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:38210
I20260810 01:02:49.291126 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 9127
I20260810 01:02:49.295287 9636 tablet_replica.cc:330] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590: stopping tablet replica
I20260810 01:02:49.295828 9636 raft_consensus.cc:2229] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590 [term 1 LEADER]: Raft consensus shutting down.
I20260810 01:02:49.296633 9636 raft_consensus.cc:2258] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:02:49.297524 9634 tablet_replica.cc:330] T 547df716a8954d53b0f717f6a68c13e0 P 60a8395609fe497b89946b00039d67f2: stopping tablet replica
I20260810 01:02:49.297938 9634 raft_consensus.cc:2229] T 547df716a8954d53b0f717f6a68c13e0 P 60a8395609fe497b89946b00039d67f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:02:49.298553 9634 raft_consensus.cc:2258] T 547df716a8954d53b0f717f6a68c13e0 P 60a8395609fe497b89946b00039d67f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:02:49.301218 9635 tablet_replica.cc:330] T 547df716a8954d53b0f717f6a68c13e0 P 5b000243ac8c40709252a1381aff4ff4: stopping tablet replica
I20260810 01:02:49.301617 9635 raft_consensus.cc:2229] T 547df716a8954d53b0f717f6a68c13e0 P 5b000243ac8c40709252a1381aff4ff4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:02:49.302122 9635 raft_consensus.cc:2258] T 547df716a8954d53b0f717f6a68c13e0 P 5b000243ac8c40709252a1381aff4ff4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:02:49.302119 9636 ts_tablet_manager.cc:1905] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:02:49.305971 9635 ts_tablet_manager.cc:1905] T 547df716a8954d53b0f717f6a68c13e0 P 5b000243ac8c40709252a1381aff4ff4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:02:49.309167 9634 ts_tablet_manager.cc:1905] T 547df716a8954d53b0f717f6a68c13e0 P 60a8395609fe497b89946b00039d67f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:02:49.310258 9636 ts_tablet_manager.cc:1918] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:02:49.310477 9636 log.cc:1196] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-0/wal/wals/547df716a8954d53b0f717f6a68c13e0
I20260810 01:02:49.311262 9636 ts_tablet_manager.cc:1939] T 547df716a8954d53b0f717f6a68c13e0 P 2d96e48a28ed43a593d77bbc96957590: Deleting consensus metadata
Aug 10 01:02:49 dist-test-slave-gp6n krb5kdc[9013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.129@KRBTEST.COM
I20260810 01:02:49.313254 9046 catalog_manager.cc:4841] TS 2d96e48a28ed43a593d77bbc96957590 (127.20.81.129:35635): tablet 547df716a8954d53b0f717f6a68c13e0 (table test-table [id=97ce7a0779fd4e3dacca9abe27d4407d]) successfully deleted
I20260810 01:02:49.314841 9635 ts_tablet_manager.cc:1918] T 547df716a8954d53b0f717f6a68c13e0 P 5b000243ac8c40709252a1381aff4ff4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:02:49.315089 9635 log.cc:1196] T 547df716a8954d53b0f717f6a68c13e0 P 5b000243ac8c40709252a1381aff4ff4: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/547df716a8954d53b0f717f6a68c13e0
I20260810 01:02:49.316051 9635 ts_tablet_manager.cc:1939] T 547df716a8954d53b0f717f6a68c13e0 P 5b000243ac8c40709252a1381aff4ff4: Deleting consensus metadata
I20260810 01:02:49.318327 9046 catalog_manager.cc:4841] TS 5b000243ac8c40709252a1381aff4ff4 (127.20.81.131:42871): tablet 547df716a8954d53b0f717f6a68c13e0 (table test-table [id=97ce7a0779fd4e3dacca9abe27d4407d]) successfully deleted
I20260810 01:02:49.319957 9634 ts_tablet_manager.cc:1918] T 547df716a8954d53b0f717f6a68c13e0 P 60a8395609fe497b89946b00039d67f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:02:49.320327 9634 log.cc:1196] T 547df716a8954d53b0f717f6a68c13e0 P 60a8395609fe497b89946b00039d67f2: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/547df716a8954d53b0f717f6a68c13e0
I20260810 01:02:49.320948 9634 ts_tablet_manager.cc:1939] T 547df716a8954d53b0f717f6a68c13e0 P 60a8395609fe497b89946b00039d67f2: Deleting consensus metadata
I20260810 01:02:49.322539 9603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:49.292426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38224 (local address 127.20.81.129:35635)
0810 01:02:49.292627 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:49.292634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:49.297782 (+ 5148us) server_negotiation.cc:407] Connection header received
0810 01:02:49.298153 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:49.298160 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:49.298242 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:49.298325 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:49.302631 (+ 4306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:49.303934 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:49.306466 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:49.307181 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:49.307295 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:49.314693 (+ 7398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:49.314709 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:49.314727 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:49.314770 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:49.318993 (+ 4223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:49.319929 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:49.319941 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:49.319952 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:49.320160 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:49.320653 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:49.320661 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:49.320667 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:49.321190 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:49.321503 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:49.321865 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 01:02:49.322139 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 01:02:49.322813 9044 catalog_manager.cc:4841] TS 60a8395609fe497b89946b00039d67f2 (127.20.81.130:33269): tablet 547df716a8954d53b0f717f6a68c13e0 (table test-table [id=97ce7a0779fd4e3dacca9abe27d4407d]) successfully deleted
I20260810 01:02:49.323201 9276 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:49.422205 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 9127
I20260810 01:02:49.445727 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 9289
Aug 10 01:02:49 dist-test-slave-gp6n krb5kdc[9013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.130@KRBTEST.COM
I20260810 01:02:49.461876 9606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:49.446678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45172 (local address 127.20.81.130:33269)
0810 01:02:49.446858 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:49.446862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:49.446881 (+ 19us) server_negotiation.cc:407] Connection header received
0810 01:02:49.447045 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:49.447048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:49.447112 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:49.447197 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:49.448494 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:49.449250 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:49.450675 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:49.451125 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:49.451237 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:49.455827 (+ 4590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:49.455843 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:49.455858 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:49.455897 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:49.459173 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:49.459921 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:49.459929 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:49.459931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:49.460069 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:49.460474 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:49.460478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:49.460480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:49.460856 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:49.461097 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:49.461431 (+ 334us) server_negotiation.cc:299] Negotiation successful
0810 01:02:49.461601 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 01:02:49.462685 9428 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:49.555698 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 9289
I20260810 01:02:49.578313 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 9439
Aug 10 01:02:49 dist-test-slave-gp6n krb5kdc[9013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.131@KRBTEST.COM
I20260810 01:02:49.595294 9608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:49.579236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51374 (local address 127.20.81.131:42871)
0810 01:02:49.579513 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:49.579520 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:49.579540 (+ 20us) server_negotiation.cc:407] Connection header received
0810 01:02:49.579715 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:49.579720 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:49.579796 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:49.579916 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:49.581251 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:49.581989 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:49.583927 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:49.584592 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:49.584751 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:49.589744 (+ 4993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:49.589760 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:49.589777 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:49.589822 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:49.592625 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:49.593284 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:49.593291 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:49.593293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:49.593449 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:49.593770 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:49.593773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:49.593775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:49.594126 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:49.594348 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:49.594723 (+ 375us) server_negotiation.cc:299] Negotiation successful
0810 01:02:49.594963 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260810 01:02:49.595960 9579 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:49.685575 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 9439
I20260810 01:02:49.706128 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 9028
I20260810 01:02:49.707520 9094 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:49.819639 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 9028
2026-08-10T01:02:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4849 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T01:02:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:02:49Z Disabled control of system clock
I20260810 01:02:49.915831 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:39101
--webserver_interface=127.20.81.190
--webserver_port=0
--builtin_ntp_servers=127.20.81.148:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:39101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 01:02:50.265187 9646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:50.265506 9646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:50.265676 9646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:50.273137 9646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:02:50.273253 9646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:50.273363 9646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:50.273445 9646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:02:50.273531 9646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:02:50.283113 9646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:39101
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:39101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:50.284547 9646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:50.286433 9646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:50.298684 9651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:50.298743 9654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:50.298717 9652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:50.299314 9646 server_base.cc:1034] running on GCE node
I20260810 01:02:50.300206 9646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:50.301662 9646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:50.302891 9646 hybrid_clock.cc:648] HybridClock initialized: now 1786323770302843 us; error 63 us; skew 500 ppm
I20260810 01:02:50.303339 9646 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:02:50.306047 9646 webserver.cc:460] Webserver started at http://127.20.81.190:36165/ using document root <none> and password file <none>
I20260810 01:02:50.306866 9646 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:50.307004 9646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:50.307333 9646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:50.310873 9646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "8b25a40ad2314d52ad14dbd168920e92"
format_stamp: "Formatted at 2026-08-10 01:02:50 on dist-test-slave-gp6n"
server_key: "ab4d1ceaf6d6317f8705ed4685c3af54"
server_key_iv: "5864ea004c0ccfa4adfad71073162bd6"
server_key_version: "encryptionkey@0"
I20260810 01:02:50.311753 9646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "8b25a40ad2314d52ad14dbd168920e92"
format_stamp: "Formatted at 2026-08-10 01:02:50 on dist-test-slave-gp6n"
server_key: "ab4d1ceaf6d6317f8705ed4685c3af54"
server_key_iv: "5864ea004c0ccfa4adfad71073162bd6"
server_key_version: "encryptionkey@0"
I20260810 01:02:50.317436 9646 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 01:02:50.321321 9666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:50.322793 9646 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 01:02:50.322997 9646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "8b25a40ad2314d52ad14dbd168920e92"
format_stamp: "Formatted at 2026-08-10 01:02:50 on dist-test-slave-gp6n"
server_key: "ab4d1ceaf6d6317f8705ed4685c3af54"
server_key_iv: "5864ea004c0ccfa4adfad71073162bd6"
server_key_version: "encryptionkey@0"
I20260810 01:02:50.323240 9646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:50.340253 9646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:50.341133 9646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:50.341439 9646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:50.361305 9646 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:39101
I20260810 01:02:50.361337 9719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:39101 every 8 connection(s)
I20260810 01:02:50.362735 9646 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:02:50.364683 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 9646
I20260810 01:02:50.364878 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:02:50.365334 20806 external_mini_cluster.cc:1428] Setting key 816736c0dcfc1b55ad2fc76cafe9857e
I20260810 01:02:50.367015 9720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:02:50.379613 9724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:50.367371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52238 (local address 127.20.81.190:39101)
0810 01:02:50.368373 (+ 1002us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:50.368392 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:50.368432 (+ 40us) server_negotiation.cc:407] Connection header received
0810 01:02:50.370026 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:50.370047 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:50.370586 (+ 539us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:50.370930 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:50.371589 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:50.373037 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:50.375065 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:50.375763 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:50.375905 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:50.377230 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:50.377237 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:50.377252 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:02:50.377825 (+ 573us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:50.377904 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:02:50.377928 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:50.378025 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:50.378258 (+ 233us) server_negotiation.cc:299] Negotiation successful
0810 01:02:50.378669 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":374,"threads_started":1}
I20260810 01:02:50.380486 9720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92: Bootstrap starting.
I20260810 01:02:50.383761 9720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92: Neither blocks nor log segments found. Creating new log.
I20260810 01:02:50.384687 9720 log.cc:824] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:50.387588 9720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92: No bootstrap required, opened a new log
I20260810 01:02:50.394680 9720 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b25a40ad2314d52ad14dbd168920e92" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39101 } }
I20260810 01:02:50.395044 9720 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:02:50.395169 9720 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b25a40ad2314d52ad14dbd168920e92, State: Initialized, Role: FOLLOWER
I20260810 01:02:50.395730 9720 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b25a40ad2314d52ad14dbd168920e92" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39101 } }
I20260810 01:02:50.395956 9720 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:02:50.396088 9720 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:02:50.396270 9720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:02:50.398373 9720 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b25a40ad2314d52ad14dbd168920e92" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39101 } }
I20260810 01:02:50.398931 9720 leader_election.cc:302] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b25a40ad2314d52ad14dbd168920e92; no voters:
I20260810 01:02:50.399485 9720 leader_election.cc:288] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:02:50.399657 9727 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:02:50.400719 9727 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92 [term 1 LEADER]: Becoming Leader. State: Replica: 8b25a40ad2314d52ad14dbd168920e92, State: Running, Role: LEADER
I20260810 01:02:50.401271 9727 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b25a40ad2314d52ad14dbd168920e92" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39101 } }
I20260810 01:02:50.401729 9720 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:02:50.404078 9728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b25a40ad2314d52ad14dbd168920e92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b25a40ad2314d52ad14dbd168920e92" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39101 } } }
I20260810 01:02:50.404680 9728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92 [sys.catalog]: This master's current role is: LEADER
I20260810 01:02:50.406862 9729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b25a40ad2314d52ad14dbd168920e92. Latest consensus state: current_term: 1 leader_uuid: "8b25a40ad2314d52ad14dbd168920e92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b25a40ad2314d52ad14dbd168920e92" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39101 } } }
I20260810 01:02:50.407294 9729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92 [sys.catalog]: This master's current role is: LEADER
I20260810 01:02:50.407294 9736 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:02:50.411808 9736 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:02:50.420385 9736 catalog_manager.cc:1347] Generated new cluster ID: 078e307817c14154bf2d76e2164fef26
I20260810 01:02:50.420521 9736 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:02:50.444542 9736 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:02:50.446390 9736 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:02:50.456189 9736 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92: Generated new TSK 0
I20260810 01:02:50.457273 9736 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:02:50.466944 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:39101
--builtin_ntp_servers=127.20.81.148:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 01:02:50.817384 9746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:50.817698 9746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:50.817787 9746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:50.825892 9746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:50.826010 9746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:50.826130 9746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 01:02:50.835846 9746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:39101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:50.837170 9746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:50.839181 9746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:50.853078 9754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:50.853148 9756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:50.853072 9752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:50.853926 9746 server_base.cc:1034] running on GCE node
I20260810 01:02:50.854638 9746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:50.855573 9746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:50.856813 9746 hybrid_clock.cc:648] HybridClock initialized: now 1786323770856744 us; error 48 us; skew 500 ppm
I20260810 01:02:50.857126 9746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:02:50.859591 9746 webserver.cc:460] Webserver started at http://127.20.81.129:33205/ using document root <none> and password file <none>
I20260810 01:02:50.860270 9746 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:50.860360 9746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:50.860626 9746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:50.863621 9746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "cc83195a91904b90bd0c279a963c98cf"
format_stamp: "Formatted at 2026-08-10 01:02:50 on dist-test-slave-gp6n"
server_key: "8b6e80df2aa86bb61c152d0fc2aa6528"
server_key_iv: "ce4f782229ea8e0aeff2c109a38136dd"
server_key_version: "encryptionkey@0"
I20260810 01:02:50.864441 9746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "cc83195a91904b90bd0c279a963c98cf"
format_stamp: "Formatted at 2026-08-10 01:02:50 on dist-test-slave-gp6n"
server_key: "8b6e80df2aa86bb61c152d0fc2aa6528"
server_key_iv: "ce4f782229ea8e0aeff2c109a38136dd"
server_key_version: "encryptionkey@0"
I20260810 01:02:50.869581 9746 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260810 01:02:50.873111 9762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:50.874583 9746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:02:50.874735 9746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "cc83195a91904b90bd0c279a963c98cf"
format_stamp: "Formatted at 2026-08-10 01:02:50 on dist-test-slave-gp6n"
server_key: "8b6e80df2aa86bb61c152d0fc2aa6528"
server_key_iv: "ce4f782229ea8e0aeff2c109a38136dd"
server_key_version: "encryptionkey@0"
I20260810 01:02:50.874927 9746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:50.906913 9746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:50.907826 9746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:50.908144 9746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:50.909158 9746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:02:50.910745 9746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:02:50.910836 9746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:50.910995 9746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:02:50.911099 9746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:50.945802 9746 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:40843
I20260810 01:02:50.945935 9887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:40843 every 8 connection(s)
I20260810 01:02:50.947274 9746 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 01:02:50.949581 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 9746
I20260810 01:02:50.949772 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 01:02:50.950433 20806 external_mini_cluster.cc:1428] Setting key a144aaf50082419c363f0725e8804f02
I20260810 01:02:50.953166 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:39101
--builtin_ntp_servers=127.20.81.148:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 01:02:50.960602 9890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:50.950034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:57601 (local address 127.20.81.190:39101)
0810 01:02:50.950593 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:50.950599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:50.951959 (+ 1360us) server_negotiation.cc:407] Connection header received
0810 01:02:50.953903 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:50.953908 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:50.953997 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:50.954116 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:50.955262 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:50.956018 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:50.957574 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:50.958003 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:50.958142 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:50.959716 (+ 1574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:50.959721 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:50.959725 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:02:50.959743 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:50.959782 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:02:50.959794 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:50.959907 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:50.960134 (+ 227us) server_negotiation.cc:299] Negotiation successful
0810 01:02:50.960238 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":274,"threads_started":1}
I20260810 01:02:50.961624 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:50.950703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:39101 (local address 127.20.81.129:57601)
0810 01:02:50.951786 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:50.951839 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:50.953639 (+ 1800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:50.954361 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:50.954369 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:50.954516 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:02:50.955092 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:50.955109 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:50.956165 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:50.956175 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:50.957421 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:50.957433 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:50.958246 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:50.958253 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:50.958562 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:50.959396 (+ 834us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 01:02:50.959399 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:02:50.959437 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 01:02:50.959512 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:02:50.959907 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:02:50.959921 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:02:50.959931 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 01:02:50.960109 (+ 178us) client_negotiation.cc:240] Negotiation successful
0810 01:02:50.960520 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":348,"threads_started":1}
I20260810 01:02:50.963270 9888 heartbeater.cc:347] Connected to a master server at 127.20.81.190:39101
I20260810 01:02:50.963598 9888 heartbeater.cc:464] Registering TS with master...
I20260810 01:02:50.964375 9888 heartbeater.cc:511] Master 127.20.81.190:39101 requested a full tablet report, sending...
I20260810 01:02:50.966743 9683 ts_manager.cc:194] Registered new tserver with Master: cc83195a91904b90bd0c279a963c98cf (127.20.81.129:40843)
I20260810 01:02:50.968262 9683 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.81.129:57601
W20260810 01:02:51.304786 9892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:51.305183 9892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:51.305362 9892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:51.312973 9892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:51.313083 9892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:51.313270 9892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 01:02:51.322820 9892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:39101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:51.324239 9892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:51.326169 9892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:51.340466 9904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:51.340582 9901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:51.340764 9902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:51.341874 9892 server_base.cc:1034] running on GCE node
I20260810 01:02:51.342566 9892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:51.343705 9892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:51.344973 9892 hybrid_clock.cc:648] HybridClock initialized: now 1786323771344970 us; error 112 us; skew 500 ppm
I20260810 01:02:51.345338 9892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:02:51.348119 9892 webserver.cc:460] Webserver started at http://127.20.81.130:41177/ using document root <none> and password file <none>
I20260810 01:02:51.348917 9892 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:51.349066 9892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:51.349439 9892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:51.352823 9892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "25e33b4a7d9742cb83ac50aefe1db060"
format_stamp: "Formatted at 2026-08-10 01:02:51 on dist-test-slave-gp6n"
server_key: "0a20b79537519e62ac0aa6372f1804f8"
server_key_iv: "019602ac1933d30a99aa74183f54c918"
server_key_version: "encryptionkey@0"
I20260810 01:02:51.353772 9892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "25e33b4a7d9742cb83ac50aefe1db060"
format_stamp: "Formatted at 2026-08-10 01:02:51 on dist-test-slave-gp6n"
server_key: "0a20b79537519e62ac0aa6372f1804f8"
server_key_iv: "019602ac1933d30a99aa74183f54c918"
server_key_version: "encryptionkey@0"
I20260810 01:02:51.359548 9892 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260810 01:02:51.364277 9911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:51.365991 9892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 01:02:51.366164 9892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "25e33b4a7d9742cb83ac50aefe1db060"
format_stamp: "Formatted at 2026-08-10 01:02:51 on dist-test-slave-gp6n"
server_key: "0a20b79537519e62ac0aa6372f1804f8"
server_key_iv: "019602ac1933d30a99aa74183f54c918"
server_key_version: "encryptionkey@0"
I20260810 01:02:51.366438 9892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:51.393610 9892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:51.394565 9892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:51.394860 9892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:51.395867 9892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:02:51.397423 9892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:02:51.397513 9892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:51.397591 9892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:02:51.397643 9892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:51.435071 9892 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:38537
I20260810 01:02:51.435168 10040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:38537 every 8 connection(s)
I20260810 01:02:51.436477 9892 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 01:02:51.439348 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 9892
I20260810 01:02:51.439579 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 01:02:51.440115 20806 external_mini_cluster.cc:1428] Setting key 200a9dbf1d7bb44886208c1d05322ed2
I20260810 01:02:51.443197 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:39101
--builtin_ntp_servers=127.20.81.148:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 01:02:51.450217 9890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:51.439264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:50915 (local address 127.20.81.190:39101)
0810 01:02:51.439503 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:51.439510 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:51.441240 (+ 1730us) server_negotiation.cc:407] Connection header received
0810 01:02:51.443257 (+ 2017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:51.443265 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:51.443377 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:51.443525 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:51.444681 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:51.445495 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:51.447279 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:51.447731 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:51.447885 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:51.449351 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:51.449356 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:51.449360 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:02:51.449379 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:51.449413 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:02:51.449426 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:51.449536 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:51.449773 (+ 237us) server_negotiation.cc:299] Negotiation successful
0810 01:02:51.449878 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 01:02:51.451356 10043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:51.439882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:39101 (local address 127.20.81.130:50915)
0810 01:02:51.441040 (+ 1158us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:51.441098 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:51.442978 (+ 1880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:51.443790 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:51.443805 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:51.443899 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:02:51.444506 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:51.444524 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:51.445655 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:51.445662 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:51.447126 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:51.447139 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:51.447981 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:51.447988 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:51.448243 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:51.448991 (+ 748us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 01:02:51.448994 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:02:51.449035 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 01:02:51.449121 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:02:51.449538 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:02:51.449553 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:02:51.449564 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 01:02:51.449763 (+ 199us) client_negotiation.cc:240] Negotiation successful
0810 01:02:51.450192 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":374,"threads_started":1}
I20260810 01:02:51.452916 10041 heartbeater.cc:347] Connected to a master server at 127.20.81.190:39101
I20260810 01:02:51.453248 10041 heartbeater.cc:464] Registering TS with master...
I20260810 01:02:51.454059 10041 heartbeater.cc:511] Master 127.20.81.190:39101 requested a full tablet report, sending...
I20260810 01:02:51.455672 9683 ts_manager.cc:194] Registered new tserver with Master: 25e33b4a7d9742cb83ac50aefe1db060 (127.20.81.130:38537)
I20260810 01:02:51.456832 9683 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.81.130:50915
W20260810 01:02:51.802983 10044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:51.803265 10044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:51.803399 10044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:51.810788 10044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:51.810885 10044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:51.811048 10044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 01:02:51.819996 10044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:39101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:51.821338 10044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:51.823138 10044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:51.837267 10052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:51.837177 10049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:51.837450 10050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:51.843271 10044 server_base.cc:1034] running on GCE node
I20260810 01:02:51.843870 10044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:51.844817 10044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:51.846036 10044 hybrid_clock.cc:648] HybridClock initialized: now 1786323771845976 us; error 59 us; skew 500 ppm
I20260810 01:02:51.846352 10044 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:02:51.848706 10044 webserver.cc:460] Webserver started at http://127.20.81.131:42187/ using document root <none> and password file <none>
I20260810 01:02:51.849422 10044 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:51.849514 10044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:51.849759 10044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:51.852705 10044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "20a7013521a247f1bb5f2ae5b1cd190d"
format_stamp: "Formatted at 2026-08-10 01:02:51 on dist-test-slave-gp6n"
server_key: "cf90a9fd95b4566a66f4cf3ff848d26a"
server_key_iv: "4960e120b5c3169ae7c4adfb76b87ec4"
server_key_version: "encryptionkey@0"
I20260810 01:02:51.853440 10044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "20a7013521a247f1bb5f2ae5b1cd190d"
format_stamp: "Formatted at 2026-08-10 01:02:51 on dist-test-slave-gp6n"
server_key: "cf90a9fd95b4566a66f4cf3ff848d26a"
server_key_iv: "4960e120b5c3169ae7c4adfb76b87ec4"
server_key_version: "encryptionkey@0"
I20260810 01:02:51.858546 10044 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260810 01:02:51.861984 10059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:51.863348 10044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:02:51.863530 10044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "20a7013521a247f1bb5f2ae5b1cd190d"
format_stamp: "Formatted at 2026-08-10 01:02:51 on dist-test-slave-gp6n"
server_key: "cf90a9fd95b4566a66f4cf3ff848d26a"
server_key_iv: "4960e120b5c3169ae7c4adfb76b87ec4"
server_key_version: "encryptionkey@0"
I20260810 01:02:51.863749 10044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:51.885751 10044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:51.886569 10044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:51.886847 10044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:51.887763 10044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:02:51.889195 10044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:02:51.889299 10044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:51.889379 10044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:02:51.889436 10044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:51.921368 10044 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:33657
I20260810 01:02:51.921459 10188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:33657 every 8 connection(s)
I20260810 01:02:51.922709 10044 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 01:02:51.929145 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 10044
I20260810 01:02:51.929391 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 01:02:51.929901 20806 external_mini_cluster.cc:1428] Setting key e5ba83d7bf9e7c404cdee515d262f840
I20260810 01:02:51.941599 9890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:51.925365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:59563 (local address 127.20.81.190:39101)
0810 01:02:51.925634 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:51.925641 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:51.927305 (+ 1664us) server_negotiation.cc:407] Connection header received
0810 01:02:51.929108 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:51.929115 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:51.929238 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:51.929364 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:51.930668 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:51.931969 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:51.933939 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:51.934461 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:51.934586 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:51.940830 (+ 6244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:51.940835 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:51.940838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:02:51.940861 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:51.940894 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:02:51.940906 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:51.941040 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:51.941220 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 01:02:51.941335 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260810 01:02:51.942544 10191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:51.926075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:39101 (local address 127.20.81.131:59563)
0810 01:02:51.927110 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:51.927153 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:51.928822 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:51.929621 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:51.929629 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:51.929719 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:02:51.930425 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:51.930454 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:51.932139 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:51.932144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:51.933798 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:51.933810 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:51.934670 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:51.934675 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:51.934883 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:51.940517 (+ 5634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 01:02:51.940520 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:02:51.940559 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 01:02:51.940644 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:02:51.940998 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:02:51.941006 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:02:51.941012 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 01:02:51.941145 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 01:02:51.941446 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":350,"threads_started":1}
I20260810 01:02:51.943907 10189 heartbeater.cc:347] Connected to a master server at 127.20.81.190:39101
I20260810 01:02:51.944285 10189 heartbeater.cc:464] Registering TS with master...
I20260810 01:02:51.945254 10189 heartbeater.cc:511] Master 127.20.81.190:39101 requested a full tablet report, sending...
I20260810 01:02:51.946921 9683 ts_manager.cc:194] Registered new tserver with Master: 20a7013521a247f1bb5f2ae5b1cd190d (127.20.81.131:33657)
I20260810 01:02:51.947832 9683 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.81.131:59563
I20260810 01:02:51.957913 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 01:02:51.968582 9890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:51.962304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40352 (local address 127.20.81.190:39101)
0810 01:02:51.962543 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:51.962548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:51.962902 (+ 354us) server_negotiation.cc:407] Connection header received
0810 01:02:51.963046 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:51.963050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:51.963121 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:51.963208 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:51.963698 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:51.964438 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:51.966027 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:51.966510 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:51.966605 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:51.967875 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:51.967879 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:51.967882 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:02:51.967922 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:51.967967 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:02:51.967999 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:51.968125 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:51.968288 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 01:02:51.968335 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 01:02:51.972093 9888 heartbeater.cc:503] Master 127.20.81.190:39101 was elected leader, sending a full tablet report...
I20260810 01:02:51.973351 9683 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 01:02:51.981269 9683 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 01:02:52.018594 10203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:51.994205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:40843 (local address 127.0.0.1:36590)
0810 01:02:52.008364 (+ 14159us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:52.008431 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:52.008731 (+ 300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:52.010359 (+ 1628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:52.010367 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:52.010433 (+ 66us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:52.010904 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:52.010917 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.012915 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.012920 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:52.015675 (+ 2755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:52.015688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.017409 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.017416 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:52.017654 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:52.017714 (+ 60us) client_negotiation.cc:769] Sending connection context
0810 01:02:52.017851 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 01:02:52.018061 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13747,"mutex_wait_us":74,"thread_start_us":14222,"threads_started":1}
I20260810 01:02:52.019430 10204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:51.994466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36590 (local address 127.20.81.129:40843)
0810 01:02:52.007476 (+ 13010us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:52.007492 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:52.009388 (+ 1896us) server_negotiation.cc:407] Connection header received
0810 01:02:52.009534 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:52.009544 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:52.009861 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:52.010066 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:52.011126 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.012742 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.015883 (+ 3141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.017183 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.017321 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:52.017636 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:52.017949 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 01:02:52.018132 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12711,"thread_start_us":14554,"threads_started":1}
I20260810 01:02:52.020077 10205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:51.994557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:33657 (local address 127.0.0.1:58712)
0810 01:02:52.009899 (+ 15342us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:52.009926 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:52.010075 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:52.011600 (+ 1525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:52.011605 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:52.011631 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:52.012047 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:52.012059 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.014084 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.014090 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:52.016236 (+ 2146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:52.016258 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.018393 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.018424 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:52.018610 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:52.018642 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 01:02:52.019650 (+ 1008us) client_negotiation.cc:240] Negotiation successful
0810 01:02:52.019787 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15171,"mutex_wait_us":51,"thread_start_us":14977,"threads_started":1}
I20260810 01:02:52.020385 10206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:51.994889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58712 (local address 127.20.81.131:33657)
0810 01:02:52.010813 (+ 15924us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:52.010825 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:52.010866 (+ 41us) server_negotiation.cc:407] Connection header received
0810 01:02:52.010974 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:52.010982 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:52.011267 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:52.011503 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:52.012236 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.013943 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.016471 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.017858 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.017996 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:52.018250 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:52.018994 (+ 744us) server_negotiation.cc:299] Negotiation successful
0810 01:02:52.019157 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15715,"thread_start_us":15538,"threads_started":1}
I20260810 01:02:52.020804 10207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:52.010214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36784 (local address 127.20.81.130:38537)
0810 01:02:52.011084 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:52.011093 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:52.011808 (+ 715us) server_negotiation.cc:407] Connection header received
0810 01:02:52.012120 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:52.012125 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:52.012394 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:52.012596 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:52.013700 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.015226 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.017506 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.019192 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.019348 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:52.019612 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:52.020260 (+ 648us) server_negotiation.cc:299] Negotiation successful
0810 01:02:52.020448 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":572,"threads_started":1}
I20260810 01:02:52.021497 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:52.011081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:38537 (local address 127.0.0.1:36784)
0810 01:02:52.011669 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:52.011693 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:52.011915 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:52.012852 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:52.012857 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:52.012885 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:52.013504 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:52.013518 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.015397 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.015403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:52.017231 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:52.017244 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.019821 (+ 2577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.019829 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:52.020043 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:52.020081 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 01:02:52.021103 (+ 1022us) client_negotiation.cc:240] Negotiation successful
0810 01:02:52.021207 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":295,"threads_started":1}
I20260810 01:02:52.023547 9816 tablet_service.cc:1467] Processing CreateTablet for tablet 768ce05f30494390830f6cc67e090162 (DEFAULT_TABLE table=test-table [id=45fcb9146eb244ff95d75f7e51cfd348]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:02:52.024829 9816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 768ce05f30494390830f6cc67e090162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:02:52.024405 10115 tablet_service.cc:1467] Processing CreateTablet for tablet 768ce05f30494390830f6cc67e090162 (DEFAULT_TABLE table=test-table [id=45fcb9146eb244ff95d75f7e51cfd348]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:02:52.025300 9965 tablet_service.cc:1467] Processing CreateTablet for tablet 768ce05f30494390830f6cc67e090162 (DEFAULT_TABLE table=test-table [id=45fcb9146eb244ff95d75f7e51cfd348]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:02:52.026329 10115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 768ce05f30494390830f6cc67e090162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:02:52.027035 9965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 768ce05f30494390830f6cc67e090162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:02:52.037138 10209 tablet_bootstrap.cc:492] T 768ce05f30494390830f6cc67e090162 P cc83195a91904b90bd0c279a963c98cf: Bootstrap starting.
I20260810 01:02:52.040220 10209 tablet_bootstrap.cc:654] T 768ce05f30494390830f6cc67e090162 P cc83195a91904b90bd0c279a963c98cf: Neither blocks nor log segments found. Creating new log.
I20260810 01:02:52.041285 10209 log.cc:824] T 768ce05f30494390830f6cc67e090162 P cc83195a91904b90bd0c279a963c98cf: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:52.042977 10211 tablet_bootstrap.cc:492] T 768ce05f30494390830f6cc67e090162 P 25e33b4a7d9742cb83ac50aefe1db060: Bootstrap starting.
I20260810 01:02:52.044620 10209 tablet_bootstrap.cc:492] T 768ce05f30494390830f6cc67e090162 P cc83195a91904b90bd0c279a963c98cf: No bootstrap required, opened a new log
I20260810 01:02:52.044821 10213 tablet_bootstrap.cc:492] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d: Bootstrap starting.
I20260810 01:02:52.044898 10209 ts_tablet_manager.cc:1397] T 768ce05f30494390830f6cc67e090162 P cc83195a91904b90bd0c279a963c98cf: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260810 01:02:52.047506 10211 tablet_bootstrap.cc:654] T 768ce05f30494390830f6cc67e090162 P 25e33b4a7d9742cb83ac50aefe1db060: Neither blocks nor log segments found. Creating new log.
I20260810 01:02:52.048161 10213 tablet_bootstrap.cc:654] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d: Neither blocks nor log segments found. Creating new log.
I20260810 01:02:52.048684 10211 log.cc:824] T 768ce05f30494390830f6cc67e090162 P 25e33b4a7d9742cb83ac50aefe1db060: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:52.049206 10213 log.cc:824] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:52.052470 10211 tablet_bootstrap.cc:492] T 768ce05f30494390830f6cc67e090162 P 25e33b4a7d9742cb83ac50aefe1db060: No bootstrap required, opened a new log
I20260810 01:02:52.052502 10213 tablet_bootstrap.cc:492] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d: No bootstrap required, opened a new log
I20260810 01:02:52.052767 10213 ts_tablet_manager.cc:1397] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260810 01:02:52.052775 10211 ts_tablet_manager.cc:1397] T 768ce05f30494390830f6cc67e090162 P 25e33b4a7d9742cb83ac50aefe1db060: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260810 01:02:52.055769 10209 raft_consensus.cc:348] T 768ce05f30494390830f6cc67e090162 P cc83195a91904b90bd0c279a963c98cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc83195a91904b90bd0c279a963c98cf" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40843 } } peers { permanent_uuid: "20a7013521a247f1bb5f2ae5b1cd190d" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33657 } } peers { permanent_uuid: "25e33b4a7d9742cb83ac50aefe1db060" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 38537 } }
I20260810 01:02:52.056229 10209 raft_consensus.cc:374] T 768ce05f30494390830f6cc67e090162 P cc83195a91904b90bd0c279a963c98cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:02:52.056391 10209 raft_consensus.cc:729] T 768ce05f30494390830f6cc67e090162 P cc83195a91904b90bd0c279a963c98cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc83195a91904b90bd0c279a963c98cf, State: Initialized, Role: FOLLOWER
I20260810 01:02:52.057062 10209 consensus_queue.cc:260] T 768ce05f30494390830f6cc67e090162 P cc83195a91904b90bd0c279a963c98cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc83195a91904b90bd0c279a963c98cf" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40843 } } peers { permanent_uuid: "20a7013521a247f1bb5f2ae5b1cd190d" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33657 } } peers { permanent_uuid: "25e33b4a7d9742cb83ac50aefe1db060" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 38537 } }
I20260810 01:02:52.058959 10209 ts_tablet_manager.cc:1428] T 768ce05f30494390830f6cc67e090162 P cc83195a91904b90bd0c279a963c98cf: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260810 01:02:52.062888 10213 raft_consensus.cc:348] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc83195a91904b90bd0c279a963c98cf" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40843 } } peers { permanent_uuid: "20a7013521a247f1bb5f2ae5b1cd190d" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33657 } } peers { permanent_uuid: "25e33b4a7d9742cb83ac50aefe1db060" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 38537 } }
I20260810 01:02:52.063027 10211 raft_consensus.cc:348] T 768ce05f30494390830f6cc67e090162 P 25e33b4a7d9742cb83ac50aefe1db060 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc83195a91904b90bd0c279a963c98cf" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40843 } } peers { permanent_uuid: "20a7013521a247f1bb5f2ae5b1cd190d" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33657 } } peers { permanent_uuid: "25e33b4a7d9742cb83ac50aefe1db060" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 38537 } }
I20260810 01:02:52.063359 10213 raft_consensus.cc:374] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:02:52.063503 10211 raft_consensus.cc:374] T 768ce05f30494390830f6cc67e090162 P 25e33b4a7d9742cb83ac50aefe1db060 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:02:52.063506 10213 raft_consensus.cc:729] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20a7013521a247f1bb5f2ae5b1cd190d, State: Initialized, Role: FOLLOWER
I20260810 01:02:52.063635 10211 raft_consensus.cc:729] T 768ce05f30494390830f6cc67e090162 P 25e33b4a7d9742cb83ac50aefe1db060 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e33b4a7d9742cb83ac50aefe1db060, State: Initialized, Role: FOLLOWER
I20260810 01:02:52.064169 10213 consensus_queue.cc:260] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc83195a91904b90bd0c279a963c98cf" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40843 } } peers { permanent_uuid: "20a7013521a247f1bb5f2ae5b1cd190d" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33657 } } peers { permanent_uuid: "25e33b4a7d9742cb83ac50aefe1db060" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 38537 } }
I20260810 01:02:52.064224 10211 consensus_queue.cc:260] T 768ce05f30494390830f6cc67e090162 P 25e33b4a7d9742cb83ac50aefe1db060 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc83195a91904b90bd0c279a963c98cf" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40843 } } peers { permanent_uuid: "20a7013521a247f1bb5f2ae5b1cd190d" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33657 } } peers { permanent_uuid: "25e33b4a7d9742cb83ac50aefe1db060" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 38537 } }
I20260810 01:02:52.065522 10189 heartbeater.cc:503] Master 127.20.81.190:39101 was elected leader, sending a full tablet report...
I20260810 01:02:52.066154 10213 ts_tablet_manager.cc:1428] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260810 01:02:52.066609 10041 heartbeater.cc:503] Master 127.20.81.190:39101 was elected leader, sending a full tablet report...
I20260810 01:02:52.066762 10211 ts_tablet_manager.cc:1428] T 768ce05f30494390830f6cc67e090162 P 25e33b4a7d9742cb83ac50aefe1db060: Time spent starting tablet: real 0.014s user 0.012s sys 0.004s
W20260810 01:02:52.174541 10190 tablet.cc:2369] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 01:02:52.188716 10042 tablet.cc:2369] T 768ce05f30494390830f6cc67e090162 P 25e33b4a7d9742cb83ac50aefe1db060: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 01:02:52.199852 9889 tablet.cc:2369] T 768ce05f30494390830f6cc67e090162 P cc83195a91904b90bd0c279a963c98cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:02:52.256275 10219 raft_consensus.cc:482] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:02:52.256665 10219 raft_consensus.cc:504] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc83195a91904b90bd0c279a963c98cf" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40843 } } peers { permanent_uuid: "20a7013521a247f1bb5f2ae5b1cd190d" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33657 } } peers { permanent_uuid: "25e33b4a7d9742cb83ac50aefe1db060" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 38537 } }
I20260810 01:02:52.258486 10219 leader_election.cc:288] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc83195a91904b90bd0c279a963c98cf (127.20.81.129:40843), 25e33b4a7d9742cb83ac50aefe1db060 (127.20.81.130:38537)
I20260810 01:02:52.264894 10204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:52.258979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:47837 (local address 127.20.81.129:40843)
0810 01:02:52.259156 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:52.259180 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:52.259202 (+ 22us) server_negotiation.cc:407] Connection header received
0810 01:02:52.259552 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:52.259560 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:52.259659 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:52.259764 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:52.260334 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.261363 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.263135 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.263933 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.264105 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:52.264181 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:52.264501 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 01:02:52.264634 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 01:02:52.264925 10191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:52.258781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:40843 (local address 127.20.81.131:47837)
0810 01:02:52.259096 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:52.259116 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:52.259250 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:52.259837 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:52.259843 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:52.259886 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:52.260171 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:52.260181 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.261515 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.261519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:52.262969 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:52.262981 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.264090 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.264096 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:52.264298 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:52.264329 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 01:02:52.264458 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 01:02:52.264625 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":1,"spinlock_wait_cycles":36608}
I20260810 01:02:52.265245 10207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:52.259108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:55369 (local address 127.20.81.130:38537)
0810 01:02:52.259376 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:52.259386 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:52.259963 (+ 577us) server_negotiation.cc:407] Connection header received
0810 01:02:52.260170 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:52.260174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:52.260299 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:52.260457 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:52.260915 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.261776 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.263592 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.264339 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.264507 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:52.264572 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:52.264880 (+ 308us) server_negotiation.cc:299] Negotiation successful
0810 01:02:52.264999 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 01:02:52.265345 10221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:52.258809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:38537 (local address 127.20.81.131:55369)
0810 01:02:52.259837 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:52.259860 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:52.259991 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:52.260470 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:52.260474 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:52.260495 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:52.260766 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:52.260772 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.261929 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.261935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:52.263389 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:52.263402 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.264507 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.264514 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:52.264721 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:52.264744 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 01:02:52.264854 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 01:02:52.264980 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":859,"mutex_wait_us":51,"thread_start_us":650,"threads_started":1}
I20260810 01:02:52.266151 9838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "768ce05f30494390830f6cc67e090162" candidate_uuid: "20a7013521a247f1bb5f2ae5b1cd190d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc83195a91904b90bd0c279a963c98cf" is_pre_election: true
I20260810 01:02:52.266538 9987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "768ce05f30494390830f6cc67e090162" candidate_uuid: "20a7013521a247f1bb5f2ae5b1cd190d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e33b4a7d9742cb83ac50aefe1db060" is_pre_election: true
I20260810 01:02:52.266800 9838 raft_consensus.cc:2454] T 768ce05f30494390830f6cc67e090162 P cc83195a91904b90bd0c279a963c98cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a7013521a247f1bb5f2ae5b1cd190d in term 0.
I20260810 01:02:52.267186 9987 raft_consensus.cc:2454] T 768ce05f30494390830f6cc67e090162 P 25e33b4a7d9742cb83ac50aefe1db060 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a7013521a247f1bb5f2ae5b1cd190d in term 0.
I20260810 01:02:52.267575 10064 leader_election.cc:302] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a7013521a247f1bb5f2ae5b1cd190d, cc83195a91904b90bd0c279a963c98cf; no voters:
I20260810 01:02:52.268036 10219 raft_consensus.cc:2781] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:02:52.268198 10219 raft_consensus.cc:482] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:02:52.268307 10219 raft_consensus.cc:3037] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:02:52.270605 10219 raft_consensus.cc:504] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc83195a91904b90bd0c279a963c98cf" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40843 } } peers { permanent_uuid: "20a7013521a247f1bb5f2ae5b1cd190d" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33657 } } peers { permanent_uuid: "25e33b4a7d9742cb83ac50aefe1db060" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 38537 } }
I20260810 01:02:52.271178 10219 leader_election.cc:288] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d [CANDIDATE]: Term 1 election: Requested vote from peers cc83195a91904b90bd0c279a963c98cf (127.20.81.129:40843), 25e33b4a7d9742cb83ac50aefe1db060 (127.20.81.130:38537)
I20260810 01:02:52.271723 9987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "768ce05f30494390830f6cc67e090162" candidate_uuid: "20a7013521a247f1bb5f2ae5b1cd190d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e33b4a7d9742cb83ac50aefe1db060"
I20260810 01:02:52.271786 9838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "768ce05f30494390830f6cc67e090162" candidate_uuid: "20a7013521a247f1bb5f2ae5b1cd190d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc83195a91904b90bd0c279a963c98cf"
I20260810 01:02:52.271972 9987 raft_consensus.cc:3037] T 768ce05f30494390830f6cc67e090162 P 25e33b4a7d9742cb83ac50aefe1db060 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:02:52.272014 9838 raft_consensus.cc:3037] T 768ce05f30494390830f6cc67e090162 P cc83195a91904b90bd0c279a963c98cf [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:02:52.274300 9987 raft_consensus.cc:2454] T 768ce05f30494390830f6cc67e090162 P 25e33b4a7d9742cb83ac50aefe1db060 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a7013521a247f1bb5f2ae5b1cd190d in term 1.
I20260810 01:02:52.274559 9838 raft_consensus.cc:2454] T 768ce05f30494390830f6cc67e090162 P cc83195a91904b90bd0c279a963c98cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a7013521a247f1bb5f2ae5b1cd190d in term 1.
I20260810 01:02:52.274772 10062 leader_election.cc:302] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a7013521a247f1bb5f2ae5b1cd190d, 25e33b4a7d9742cb83ac50aefe1db060; no voters:
I20260810 01:02:52.275079 10219 raft_consensus.cc:2781] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:02:52.275530 10219 raft_consensus.cc:686] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d [term 1 LEADER]: Becoming Leader. State: Replica: 20a7013521a247f1bb5f2ae5b1cd190d, State: Running, Role: LEADER
I20260810 01:02:52.276014 10219 consensus_queue.cc:237] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc83195a91904b90bd0c279a963c98cf" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40843 } } peers { permanent_uuid: "20a7013521a247f1bb5f2ae5b1cd190d" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33657 } } peers { permanent_uuid: "25e33b4a7d9742cb83ac50aefe1db060" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 38537 } }
I20260810 01:02:52.280611 9683 catalog_manager.cc:5495] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d reported cstate change: term changed from 0 to 1, leader changed from <none> to 20a7013521a247f1bb5f2ae5b1cd190d (127.20.81.131). New cstate: current_term: 1 leader_uuid: "20a7013521a247f1bb5f2ae5b1cd190d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc83195a91904b90bd0c279a963c98cf" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a7013521a247f1bb5f2ae5b1cd190d" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 33657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25e33b4a7d9742cb83ac50aefe1db060" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 38537 } health_report { overall_health: UNKNOWN } } }
I20260810 01:02:52.305488 10206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:52.299495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58724 (local address 127.20.81.131:33657)
0810 01:02:52.299720 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:52.299726 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:52.299839 (+ 113us) server_negotiation.cc:407] Connection header received
0810 01:02:52.300097 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:52.300101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:52.300182 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:52.300342 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:02:52.300761 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.301578 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.303117 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.303534 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.303676 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:52.304321 (+ 645us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:02:52.304846 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:02:52.305049 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:52.305163 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 01:02:52.305206 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 01:02:52.314118 9838 raft_consensus.cc:1261] T 768ce05f30494390830f6cc67e090162 P cc83195a91904b90bd0c279a963c98cf [term 1 FOLLOWER]: Refusing update from remote peer 20a7013521a247f1bb5f2ae5b1cd190d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:02:52.314186 9987 raft_consensus.cc:1261] T 768ce05f30494390830f6cc67e090162 P 25e33b4a7d9742cb83ac50aefe1db060 [term 1 FOLLOWER]: Refusing update from remote peer 20a7013521a247f1bb5f2ae5b1cd190d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:02:52.315183 10222 consensus_queue.cc:1035] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e33b4a7d9742cb83ac50aefe1db060" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 38537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:02:52.315551 10219 consensus_queue.cc:1035] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc83195a91904b90bd0c279a963c98cf" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:02:52.328012 10228 mvcc.cc:205] Tried to move back new op lower bound from 7316782171388440576 to 7316782171246092288. Current Snapshot: MvccSnapshot[applied={T|T < 7316782171388440576}]
I20260810 01:02:52.344120 9682 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:02:52.344480 9682 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:02:52.348699 9682 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b25a40ad2314d52ad14dbd168920e92: Sending DeleteTablet for 3 replicas of tablet 768ce05f30494390830f6cc67e090162
I20260810 01:02:52.349896 9816 tablet_service.cc:1514] Processing DeleteTablet for tablet 768ce05f30494390830f6cc67e090162 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:02:52 UTC) from {username='slave'} at 127.0.0.1:36590
I20260810 01:02:52.350046 9965 tablet_service.cc:1514] Processing DeleteTablet for tablet 768ce05f30494390830f6cc67e090162 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:02:52 UTC) from {username='slave'} at 127.0.0.1:36784
I20260810 01:02:52.350975 10115 tablet_service.cc:1514] Processing DeleteTablet for tablet 768ce05f30494390830f6cc67e090162 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:02:52 UTC) from {username='slave'} at 127.0.0.1:58712
I20260810 01:02:52.351102 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 9746
I20260810 01:02:52.370376 10240 tablet_replica.cc:330] T 768ce05f30494390830f6cc67e090162 P 25e33b4a7d9742cb83ac50aefe1db060: stopping tablet replica
I20260810 01:02:52.370739 10241 tablet_replica.cc:330] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d: stopping tablet replica
I20260810 01:02:52.370980 10240 raft_consensus.cc:2229] T 768ce05f30494390830f6cc67e090162 P 25e33b4a7d9742cb83ac50aefe1db060 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:02:52.371372 10241 raft_consensus.cc:2229] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d [term 1 LEADER]: Raft consensus shutting down.
I20260810 01:02:52.372206 10241 raft_consensus.cc:2258] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:02:52.372671 10240 raft_consensus.cc:2258] T 768ce05f30494390830f6cc67e090162 P 25e33b4a7d9742cb83ac50aefe1db060 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:02:52.374831 10241 ts_tablet_manager.cc:1905] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:02:52.375485 10239 tablet_replica.cc:330] T 768ce05f30494390830f6cc67e090162 P cc83195a91904b90bd0c279a963c98cf: stopping tablet replica
I20260810 01:02:52.375917 10239 raft_consensus.cc:2229] T 768ce05f30494390830f6cc67e090162 P cc83195a91904b90bd0c279a963c98cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:02:52.376492 10239 raft_consensus.cc:2258] T 768ce05f30494390830f6cc67e090162 P cc83195a91904b90bd0c279a963c98cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:02:52.378876 10240 ts_tablet_manager.cc:1905] T 768ce05f30494390830f6cc67e090162 P 25e33b4a7d9742cb83ac50aefe1db060: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:02:52.384188 10241 ts_tablet_manager.cc:1918] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:02:52.377398 10204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:52.352625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36598 (local address 127.20.81.129:40843)
0810 01:02:52.352848 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:52.352858 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:52.364312 (+ 11454us) server_negotiation.cc:407] Connection header received
0810 01:02:52.364839 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:52.364848 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:52.364993 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:52.365082 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:52.365880 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.367328 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.371235 (+ 3907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.371869 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.371982 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:52.375489 (+ 3507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:52.375500 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:52.375519 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:02:52.376291 (+ 772us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:52.376357 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:02:52.376401 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:52.376502 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:52.376883 (+ 381us) server_negotiation.cc:299] Negotiation successful
0810 01:02:52.376998 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 01:02:52.385103 9883 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:52.386395 10240 ts_tablet_manager.cc:1918] T 768ce05f30494390830f6cc67e090162 P 25e33b4a7d9742cb83ac50aefe1db060: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:02:52.386708 10240 log.cc:1196] T 768ce05f30494390830f6cc67e090162 P 25e33b4a7d9742cb83ac50aefe1db060: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/768ce05f30494390830f6cc67e090162
I20260810 01:02:52.387509 10240 ts_tablet_manager.cc:1939] T 768ce05f30494390830f6cc67e090162 P 25e33b4a7d9742cb83ac50aefe1db060: Deleting consensus metadata
I20260810 01:02:52.388823 10241 log.cc:1196] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/768ce05f30494390830f6cc67e090162
I20260810 01:02:52.389467 10241 ts_tablet_manager.cc:1939] T 768ce05f30494390830f6cc67e090162 P 20a7013521a247f1bb5f2ae5b1cd190d: Deleting consensus metadata
I20260810 01:02:52.390005 9668 catalog_manager.cc:4841] TS 25e33b4a7d9742cb83ac50aefe1db060 (127.20.81.130:38537): tablet 768ce05f30494390830f6cc67e090162 (table test-table [id=45fcb9146eb244ff95d75f7e51cfd348]) successfully deleted
I20260810 01:02:52.391243 9668 catalog_manager.cc:4841] TS 20a7013521a247f1bb5f2ae5b1cd190d (127.20.81.131:33657): tablet 768ce05f30494390830f6cc67e090162 (table test-table [id=45fcb9146eb244ff95d75f7e51cfd348]) successfully deleted
I20260810 01:02:52.492617 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 9746
W20260810 01:02:52.513576 9670 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.81.129:40843 (error 108)
I20260810 01:02:52.514066 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 9892
I20260810 01:02:52.515525 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:52.514942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:40843 (local address 127.0.0.1:36612)
0810 01:02:52.515204 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:52.515372 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:40843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
W20260810 01:02:52.515790 9670 catalog_manager.cc:4584] TS cc83195a91904b90bd0c279a963c98cf (127.20.81.129:40843): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 768ce05f30494390830f6cc67e090162: Network error: Client connection negotiation failed: client connection to 127.20.81.129:40843: connect: Connection refused (error 111)
I20260810 01:02:52.522768 10207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:52.515236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36792 (local address 127.20.81.130:38537)
0810 01:02:52.515428 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:52.515435 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:52.515454 (+ 19us) server_negotiation.cc:407] Connection header received
0810 01:02:52.515523 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:52.515528 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:52.515635 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:52.515741 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:52.516437 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.517213 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.519214 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.519670 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.519774 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:52.521115 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:52.521124 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:52.521139 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:02:52.521830 (+ 691us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:52.521886 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:02:52.521908 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:52.522013 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:52.522186 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 01:02:52.522281 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 01:02:52.523474 10036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:52.569626 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:52.569078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:40843 (local address 127.0.0.1:36622)
0810 01:02:52.569360 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:52.569455 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:40843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 01:02:52.570923 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:52.570525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:40843 (local address 127.0.0.1:36632)
0810 01:02:52.570700 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:52.570804 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:40843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 01:02:52.631760 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:52.631163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:40843 (local address 127.0.0.1:36644)
0810 01:02:52.631449 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:52.631550 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:40843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260810 01:02:52.632997 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:52.632604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:40843 (local address 127.0.0.1:36650)
0810 01:02:52.632781 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:52.632867 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:40843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 01:02:52.667289 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 9892
I20260810 01:02:52.689841 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 10044
I20260810 01:02:52.698148 10206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:52.690787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58732 (local address 127.20.81.131:33657)
0810 01:02:52.690989 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:52.690997 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:52.691090 (+ 93us) server_negotiation.cc:407] Connection header received
0810 01:02:52.691330 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:52.691336 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:52.691453 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:52.691593 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:52.692071 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.692978 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.694580 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:52.695112 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:52.695275 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:52.696548 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:52.696558 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:52.696575 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:02:52.697272 (+ 697us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:52.697330 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:02:52.697353 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:52.697518 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:52.697653 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 01:02:52.697762 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 01:02:52.698954 10184 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:52.713496 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:52.713002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:40843 (local address 127.0.0.1:36652)
0810 01:02:52.713220 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:52.713310 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:40843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 01:02:52.714617 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:52.714155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:40843 (local address 127.0.0.1:36662)
0810 01:02:52.714364 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:52.714508 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:40843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260810 01:02:52.823599 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 10044
I20260810 01:02:52.849284 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 9646
I20260810 01:02:52.850469 9715 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:52.962641 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:52.961997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:40843 (local address 127.0.0.1:36668)
0810 01:02:52.962260 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:52.962365 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:40843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260810 01:02:52.962716 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 9646
2026-08-10T01:02:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3142 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:02:53 dist-test-slave-gp6n krb5kdc[10251](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:02:53 dist-test-slave-gp6n krb5kdc[10251](info): set up 2 sockets
Aug 10 01:02:53 dist-test-slave-gp6n krb5kdc[10251](info): commencing operation
krb5kdc: starting...
W20260810 01:02:54.207962 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:02:54 dist-test-slave-gp6n krb5kdc[10251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:02:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:02:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 01:02:54.550551 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:43537
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:43537
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:02:54.904497 10264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:54.904809 10264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:54.904903 10264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:54.912314 10264 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:02:54.912427 10264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:54.912473 10264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:54.912532 10264 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:02:54.912582 10264 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:02:54.922394 10264 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:43537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:43537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:54.923719 10264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:54.925019 10264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:54.936903 10272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:54.936836 10270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:54.937336 10269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:54.937989 10264 server_base.cc:1034] running on GCE node
I20260810 01:02:54.938797 10264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:54.940166 10264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:54.941367 10264 hybrid_clock.cc:648] HybridClock initialized: now 1786323774941339 us; error 53 us; skew 500 ppm
I20260810 01:02:54.941665 10264 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:02:54 dist-test-slave-gp6n krb5kdc[10251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323774, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:02:54.944787 10264 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 01:02:54.946357 10264 webserver.cc:460] Webserver started at http://127.20.81.190:41625/ using document root <none> and password file <none>
I20260810 01:02:54.947088 10264 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:54.947242 10264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:54.947557 10264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:54.950695 10264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "bf8f48e8cbcc412f87ac325c278cf559"
format_stamp: "Formatted at 2026-08-10 01:02:54 on dist-test-slave-gp6n"
server_key: "4a93a7fedf0166d5b2d9c83f2f08cedc"
server_key_iv: "5dd5eecec53153b969c3cb47a3f5d2df"
server_key_version: "encryptionkey@0"
I20260810 01:02:54.951480 10264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "bf8f48e8cbcc412f87ac325c278cf559"
format_stamp: "Formatted at 2026-08-10 01:02:54 on dist-test-slave-gp6n"
server_key: "4a93a7fedf0166d5b2d9c83f2f08cedc"
server_key_iv: "5dd5eecec53153b969c3cb47a3f5d2df"
server_key_version: "encryptionkey@0"
I20260810 01:02:54.956861 10264 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 01:02:54.960445 10281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:54.961730 10264 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 01:02:54.961915 10264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "bf8f48e8cbcc412f87ac325c278cf559"
format_stamp: "Formatted at 2026-08-10 01:02:54 on dist-test-slave-gp6n"
server_key: "4a93a7fedf0166d5b2d9c83f2f08cedc"
server_key_iv: "5dd5eecec53153b969c3cb47a3f5d2df"
server_key_version: "encryptionkey@0"
I20260810 01:02:54.962133 10264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:54.986279 10264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:54.990478 10264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:54.990785 10264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:55.010978 10264 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:43537
I20260810 01:02:55.011003 10338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:43537 every 8 connection(s)
I20260810 01:02:55.012471 10264 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:02:55.013932 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 10264
I20260810 01:02:55.014143 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:02:55.014804 20806 external_mini_cluster.cc:1428] Setting key 60b98dd4f52b4cff98f3e2150522e4f6
I20260810 01:02:55.017446 10339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:02:55.028489 10339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559: Bootstrap starting.
Aug 10 01:02:55 dist-test-slave-gp6n krb5kdc[10251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:02:55.032299 10339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559: Neither blocks nor log segments found. Creating new log.
I20260810 01:02:55.033524 10339 log.cc:824] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:55.036659 10339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559: No bootstrap required, opened a new log
I20260810 01:02:55.038453 10343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:55.016827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43596 (local address 127.20.81.190:43537)
0810 01:02:55.017677 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:55.017689 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:55.017734 (+ 45us) server_negotiation.cc:407] Connection header received
0810 01:02:55.019528 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:55.019550 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:55.019938 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:55.020356 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:55.021874 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:55.023268 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:55.024627 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:55.025335 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:55.025560 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:55.030166 (+ 4606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:55.030173 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:55.030189 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:55.030231 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:55.033908 (+ 3677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:55.034844 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:55.034851 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:55.034858 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:55.035069 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:55.035482 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:55.035488 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:55.035491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:55.036056 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:55.036363 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:55.036862 (+ 499us) server_negotiation.cc:299] Negotiation successful
0810 01:02:55.037352 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":341,"threads_started":1}
I20260810 01:02:55.045012 10339 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8f48e8cbcc412f87ac325c278cf559" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43537 } }
I20260810 01:02:55.045441 10339 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:02:55.045560 10339 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf8f48e8cbcc412f87ac325c278cf559, State: Initialized, Role: FOLLOWER
I20260810 01:02:55.046295 10339 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8f48e8cbcc412f87ac325c278cf559" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43537 } }
I20260810 01:02:55.046612 10339 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:02:55.046777 10339 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:02:55.046981 10339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:02:55.049238 10339 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8f48e8cbcc412f87ac325c278cf559" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43537 } }
I20260810 01:02:55.049734 10339 leader_election.cc:302] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf8f48e8cbcc412f87ac325c278cf559; no voters:
I20260810 01:02:55.050287 10339 leader_election.cc:288] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:02:55.050478 10345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:02:55.051296 10345 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559 [term 1 LEADER]: Becoming Leader. State: Replica: bf8f48e8cbcc412f87ac325c278cf559, State: Running, Role: LEADER
I20260810 01:02:55.051895 10345 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8f48e8cbcc412f87ac325c278cf559" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43537 } }
I20260810 01:02:55.052659 10339 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:02:55.055262 10346 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf8f48e8cbcc412f87ac325c278cf559" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8f48e8cbcc412f87ac325c278cf559" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43537 } } }
I20260810 01:02:55.055241 10347 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf8f48e8cbcc412f87ac325c278cf559. Latest consensus state: current_term: 1 leader_uuid: "bf8f48e8cbcc412f87ac325c278cf559" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8f48e8cbcc412f87ac325c278cf559" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43537 } } }
I20260810 01:02:55.055720 10346 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559 [sys.catalog]: This master's current role is: LEADER
I20260810 01:02:55.055720 10347 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559 [sys.catalog]: This master's current role is: LEADER
I20260810 01:02:55.058298 10354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:02:55.063927 10354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:02:55.072811 10354 catalog_manager.cc:1347] Generated new cluster ID: fd7b17e604a54bc5baf22ff70efa5df2
I20260810 01:02:55.072934 10354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:02:55.087162 10354 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:02:55.088507 10354 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:02:55.097636 10354 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559: Generated new TSK 0
I20260810 01:02:55.098743 10354 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.81.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
Entry for principal kudu/127.20.81.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab.
I20260810 01:02:55.182513 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:43537
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:02:55.553435 10370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:55.553707 10370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:55.553879 10370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:55.561427 10370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:55.561576 10370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:55.561760 10370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 01:02:55.571923 10370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.129.keytab
--principal=kudu/127.20.81.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:43537
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:55.573225 10370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:55.574615 10370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:55.588063 10379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:55.588191 10378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:55.588609 10381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:55.588938 10370 server_base.cc:1034] running on GCE node
I20260810 01:02:55.589556 10370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:55.590577 10370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:55.591813 10370 hybrid_clock.cc:648] HybridClock initialized: now 1786323775591775 us; error 47 us; skew 500 ppm
I20260810 01:02:55.592140 10370 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:02:55 dist-test-slave-gp6n krb5kdc[10251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323775, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:02:55.595428 10370 init.cc:377] Logged in from keytab as kudu/127.20.81.129@KRBTEST.COM (short username kudu)
I20260810 01:02:55.596943 10370 webserver.cc:460] Webserver started at http://127.20.81.129:45269/ using document root <none> and password file <none>
I20260810 01:02:55.597649 10370 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:55.597805 10370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:55.598129 10370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:55.601333 10370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "95ea9e4492c84655906846f862cd0585"
format_stamp: "Formatted at 2026-08-10 01:02:55 on dist-test-slave-gp6n"
server_key: "0fd16aa790fd40bc23e99640c9f5d513"
server_key_iv: "a00cb24a9fa551b34f754e53be7cae42"
server_key_version: "encryptionkey@0"
I20260810 01:02:55.602124 10370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "95ea9e4492c84655906846f862cd0585"
format_stamp: "Formatted at 2026-08-10 01:02:55 on dist-test-slave-gp6n"
server_key: "0fd16aa790fd40bc23e99640c9f5d513"
server_key_iv: "a00cb24a9fa551b34f754e53be7cae42"
server_key_version: "encryptionkey@0"
I20260810 01:02:55.608506 10370 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260810 01:02:55.612547 10388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:55.614130 10370 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 01:02:55.614331 10370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "95ea9e4492c84655906846f862cd0585"
format_stamp: "Formatted at 2026-08-10 01:02:55 on dist-test-slave-gp6n"
server_key: "0fd16aa790fd40bc23e99640c9f5d513"
server_key_iv: "a00cb24a9fa551b34f754e53be7cae42"
server_key_version: "encryptionkey@0"
I20260810 01:02:55.614657 10370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:55.636487 10370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:55.640832 10370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:55.641140 10370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:55.642163 10370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:02:55.643558 10370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:02:55.643667 10370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:55.643836 10370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:02:55.643914 10370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:55.679032 10370 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:45543
I20260810 01:02:55.679065 10516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:45543 every 8 connection(s)
I20260810 01:02:55.680436 10370 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 01:02:55.687896 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 10370
I20260810 01:02:55.688089 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 01:02:55.688669 20806 external_mini_cluster.cc:1428] Setting key 25fb408dbad76a9609c3bc6ae3dfff39
Aug 10 01:02:55 dist-test-slave-gp6n krb5kdc[10251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323775, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.129@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:02:55.703189 10520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:55.683141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:57857 (local address 127.20.81.190:43537)
0810 01:02:55.683635 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:55.683640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:55.684832 (+ 1192us) server_negotiation.cc:407] Connection header received
0810 01:02:55.686676 (+ 1844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:55.686681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:55.686765 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:55.686871 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:55.688476 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:55.689490 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:55.691348 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:55.692053 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:55.692214 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:55.696067 (+ 3853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:55.696075 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:55.696083 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:55.696128 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:55.699724 (+ 3596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:55.700509 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:55.700517 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:55.700522 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:55.700681 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:55.701157 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:55.701164 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:55.701170 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:55.701557 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:55.701710 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:55.702428 (+ 718us) server_negotiation.cc:299] Negotiation successful
0810 01:02:55.702710 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":257,"threads_started":1}
I20260810 01:02:55.703929 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:55.683888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:43537 (local address 127.20.81.129:57857)
0810 01:02:55.684644 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:55.684690 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:55.686362 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:55.687068 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:55.687075 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:55.687631 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:02:55.688290 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:55.688323 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:55.689696 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:55.689700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:55.691178 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:55.691192 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:55.692344 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:55.692351 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:55.692579 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:55.693320 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:02:55.693342 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:02:55.695842 (+ 2500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:02:55.699885 (+ 4043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:02:55.699892 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:02:55.699904 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:02:55.700367 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:02:55.700801 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:02:55.700804 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:02:55.700807 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:02:55.701033 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:02:55.701679 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:02:55.701684 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:02:55.702001 (+ 317us) client_negotiation.cc:769] Sending connection context
0810 01:02:55.702220 (+ 219us) client_negotiation.cc:240] Negotiation successful
0810 01:02:55.702792 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":284,"threads_started":1}
I20260810 01:02:55.705722 10517 heartbeater.cc:347] Connected to a master server at 127.20.81.190:43537
I20260810 01:02:55.706054 10517 heartbeater.cc:464] Registering TS with master...
I20260810 01:02:55.706813 10517 heartbeater.cc:511] Master 127.20.81.190:43537 requested a full tablet report, sending...
I20260810 01:02:55.708895 10302 ts_manager.cc:194] Registered new tserver with Master: 95ea9e4492c84655906846f862cd0585 (127.20.81.129:45543)
I20260810 01:02:55.710551 10302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.129@KRBTEST.COM'} at 127.20.81.129:57857
WARNING: no policy specified for kudu/127.20.81.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
Entry for principal kudu/127.20.81.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab.
I20260810 01:02:55.762285 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:43537
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:02:56.123052 10524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:56.123380 10524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:56.123562 10524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:56.130713 10524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:56.130865 10524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:56.131060 10524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 01:02:56.140748 10524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.130.keytab
--principal=kudu/127.20.81.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:43537
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:56.142078 10524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:56.143483 10524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:56.157222 10531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:56.157387 10535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:56.157387 10532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:56.158176 10524 server_base.cc:1034] running on GCE node
I20260810 01:02:56.158844 10524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:56.159768 10524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:56.161006 10524 hybrid_clock.cc:648] HybridClock initialized: now 1786323776160965 us; error 53 us; skew 500 ppm
I20260810 01:02:56.161322 10524 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:02:56 dist-test-slave-gp6n krb5kdc[10251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323776, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:02:56.164602 10524 init.cc:377] Logged in from keytab as kudu/127.20.81.130@KRBTEST.COM (short username kudu)
I20260810 01:02:56.166124 10524 webserver.cc:460] Webserver started at http://127.20.81.130:45465/ using document root <none> and password file <none>
I20260810 01:02:56.166889 10524 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:56.167042 10524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:56.167366 10524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:56.170275 10524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "034384f6897546358c09883c8053cd2c"
format_stamp: "Formatted at 2026-08-10 01:02:56 on dist-test-slave-gp6n"
server_key: "f466d2d0b03b4cef97f81ceb34cf92d2"
server_key_iv: "dd0881e1e536f63cce9aa5ebf310133b"
server_key_version: "encryptionkey@0"
I20260810 01:02:56.171186 10524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "034384f6897546358c09883c8053cd2c"
format_stamp: "Formatted at 2026-08-10 01:02:56 on dist-test-slave-gp6n"
server_key: "f466d2d0b03b4cef97f81ceb34cf92d2"
server_key_iv: "dd0881e1e536f63cce9aa5ebf310133b"
server_key_version: "encryptionkey@0"
I20260810 01:02:56.176473 10524 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260810 01:02:56.180176 10544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:56.181540 10524 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 01:02:56.181718 10524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "034384f6897546358c09883c8053cd2c"
format_stamp: "Formatted at 2026-08-10 01:02:56 on dist-test-slave-gp6n"
server_key: "f466d2d0b03b4cef97f81ceb34cf92d2"
server_key_iv: "dd0881e1e536f63cce9aa5ebf310133b"
server_key_version: "encryptionkey@0"
I20260810 01:02:56.181934 10524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:56.197628 10524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:56.201704 10524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:56.201992 10524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:56.203058 10524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:02:56.204474 10524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:02:56.204573 10524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:56.204732 10524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:02:56.204798 10524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:56.243045 10524 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:42157
I20260810 01:02:56.243093 10677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:42157 every 8 connection(s)
I20260810 01:02:56.244324 10524 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 01:02:56.246975 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 10524
I20260810 01:02:56.247197 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 01:02:56.247830 20806 external_mini_cluster.cc:1428] Setting key de4cf8fa9a1166c5bdd236c11ee5b8f8
Aug 10 01:02:56 dist-test-slave-gp6n krb5kdc[10251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323776, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.130@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:02:56.265699 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:56.247284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:49633 (local address 127.20.81.190:43537)
0810 01:02:56.247783 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:56.247790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:56.248876 (+ 1086us) server_negotiation.cc:407] Connection header received
0810 01:02:56.250834 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:56.250841 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:56.250932 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:56.251054 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:56.252552 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:56.253402 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:56.255202 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:56.255653 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:56.255792 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:56.259551 (+ 3759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:56.259555 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:56.259559 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:56.259591 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:56.262261 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:56.263015 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:56.263025 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:56.263027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:56.263153 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:56.263714 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:56.263718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:56.263720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:56.263995 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:56.264113 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:56.265048 (+ 935us) server_negotiation.cc:299] Negotiation successful
0810 01:02:56.265278 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":236,"threads_started":1}
I20260810 01:02:56.266884 10682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:56.247854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:43537 (local address 127.20.81.130:49633)
0810 01:02:56.248730 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:56.248787 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:56.250572 (+ 1785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:56.251291 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:56.251299 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:56.251806 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:02:56.252382 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:56.252399 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:56.253587 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:56.253592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:56.255044 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:56.255057 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:56.255901 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:56.255910 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:56.256148 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:56.256892 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:02:56.256915 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:02:56.259310 (+ 2395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:02:56.262429 (+ 3119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:02:56.262437 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:02:56.262454 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:02:56.262875 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:02:56.263270 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:02:56.263275 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:02:56.263279 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:02:56.263568 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:02:56.264129 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:02:56.264138 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:02:56.264598 (+ 460us) client_negotiation.cc:769] Sending connection context
0810 01:02:56.264940 (+ 342us) client_negotiation.cc:240] Negotiation successful
0810 01:02:56.265501 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":361,"threads_started":1}
I20260810 01:02:56.268500 10678 heartbeater.cc:347] Connected to a master server at 127.20.81.190:43537
I20260810 01:02:56.268810 10678 heartbeater.cc:464] Registering TS with master...
I20260810 01:02:56.269557 10678 heartbeater.cc:511] Master 127.20.81.190:43537 requested a full tablet report, sending...
I20260810 01:02:56.271152 10302 ts_manager.cc:194] Registered new tserver with Master: 034384f6897546358c09883c8053cd2c (127.20.81.130:42157)
I20260810 01:02:56.272464 10302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.130@KRBTEST.COM'} at 127.20.81.130:49633
WARNING: no policy specified for kudu/127.20.81.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
Entry for principal kudu/127.20.81.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab.
I20260810 01:02:56.322301 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:43537
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:02:56.676676 10686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:56.677012 10686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:56.677191 10686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:56.684800 10686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:56.684952 10686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:56.685124 10686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 01:02:56.694893 10686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.131.keytab
--principal=kudu/127.20.81.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:43537
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:56.696194 10686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:56.697531 10686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:56.710950 10698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:56.711018 10694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:56.711045 10693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:56.711894 10686 server_base.cc:1034] running on GCE node
I20260810 01:02:56.712559 10686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:56.713526 10517 heartbeater.cc:503] Master 127.20.81.190:43537 was elected leader, sending a full tablet report...
I20260810 01:02:56.713599 10686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:56.714830 10686 hybrid_clock.cc:648] HybridClock initialized: now 1786323776714769 us; error 31 us; skew 500 ppm
I20260810 01:02:56.715241 10686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:02:56 dist-test-slave-gp6n krb5kdc[10251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323776, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:02:56.718621 10686 init.cc:377] Logged in from keytab as kudu/127.20.81.131@KRBTEST.COM (short username kudu)
I20260810 01:02:56.720178 10686 webserver.cc:460] Webserver started at http://127.20.81.131:45409/ using document root <none> and password file <none>
I20260810 01:02:56.720862 10686 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:56.721022 10686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:56.721349 10686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:56.724308 10686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "674bdeedd3934b77b4b34b5ca949db8b"
format_stamp: "Formatted at 2026-08-10 01:02:56 on dist-test-slave-gp6n"
server_key: "768c02dc4fba1ad988fb4d881202abd7"
server_key_iv: "c4ed63decc52518660b645a4afc76896"
server_key_version: "encryptionkey@0"
I20260810 01:02:56.725178 10686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "674bdeedd3934b77b4b34b5ca949db8b"
format_stamp: "Formatted at 2026-08-10 01:02:56 on dist-test-slave-gp6n"
server_key: "768c02dc4fba1ad988fb4d881202abd7"
server_key_iv: "c4ed63decc52518660b645a4afc76896"
server_key_version: "encryptionkey@0"
I20260810 01:02:56.730463 10686 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 01:02:56.733894 10705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:56.735342 10686 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 01:02:56.735510 10686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "674bdeedd3934b77b4b34b5ca949db8b"
format_stamp: "Formatted at 2026-08-10 01:02:56 on dist-test-slave-gp6n"
server_key: "768c02dc4fba1ad988fb4d881202abd7"
server_key_iv: "c4ed63decc52518660b645a4afc76896"
server_key_version: "encryptionkey@0"
I20260810 01:02:56.735725 10686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:56.769147 10686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:56.773828 10686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:56.774156 10686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:56.775581 10686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:02:56.776990 10686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:02:56.777097 10686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:56.777235 10686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:02:56.777311 10686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:56.811980 10686 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:38261
I20260810 01:02:56.812036 10830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:38261 every 8 connection(s)
I20260810 01:02:56.813325 10686 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 01:02:56.816781 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 10686
I20260810 01:02:56.817004 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 01:02:56.817543 20806 external_mini_cluster.cc:1428] Setting key 5ca628f6659030f3a2d167a2382881fd
Aug 10 01:02:56 dist-test-slave-gp6n krb5kdc[10251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323776, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.131@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:02:56.833848 10834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:56.815874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:42495 (local address 127.20.81.190:43537)
0810 01:02:56.816471 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:56.816478 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:56.817415 (+ 937us) server_negotiation.cc:407] Connection header received
0810 01:02:56.819746 (+ 2331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:56.819750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:56.819830 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:56.819919 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:56.821621 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:56.822387 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:56.824212 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:56.824629 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:56.824735 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:56.828243 (+ 3508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:56.828247 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:56.828250 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:56.828281 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:56.830769 (+ 2488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:56.831432 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:56.831437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:56.831440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:56.831598 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:56.832104 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:56.832110 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:56.832114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:56.832476 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:56.832613 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:56.833401 (+ 788us) server_negotiation.cc:299] Negotiation successful
0810 01:02:56.833584 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":301,"threads_started":1}
I20260810 01:02:56.834803 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:56.816430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:43537 (local address 127.20.81.131:42495)
0810 01:02:56.817240 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:56.817287 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:56.819460 (+ 2173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:56.820200 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:56.820211 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:56.820907 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:02:56.821471 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:56.821488 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:56.822594 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:56.822598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:56.823992 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:56.824006 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:56.824894 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:56.824899 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:56.825123 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:56.825840 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:02:56.825867 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:02:56.828066 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:02:56.830908 (+ 2842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:02:56.830914 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:02:56.830924 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:02:56.831330 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:02:56.831767 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:02:56.831772 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:02:56.831776 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:02:56.831987 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:02:56.832620 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:02:56.832628 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:02:56.833081 (+ 453us) client_negotiation.cc:769] Sending connection context
0810 01:02:56.833309 (+ 228us) client_negotiation.cc:240] Negotiation successful
0810 01:02:56.833672 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":357,"threads_started":1}
I20260810 01:02:56.836144 10831 heartbeater.cc:347] Connected to a master server at 127.20.81.190:43537
I20260810 01:02:56.836444 10831 heartbeater.cc:464] Registering TS with master...
I20260810 01:02:56.837234 10831 heartbeater.cc:511] Master 127.20.81.190:43537 requested a full tablet report, sending...
I20260810 01:02:56.838886 10302 ts_manager.cc:194] Registered new tserver with Master: 674bdeedd3934b77b4b34b5ca949db8b (127.20.81.131:38261)
I20260810 01:02:56.839705 10302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.81.131@KRBTEST.COM'} at 127.20.81.131:42495
I20260810 01:02:56.845008 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 01:02:56.861830 10834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:56.849328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43606 (local address 127.20.81.190:43537)
0810 01:02:56.849496 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:56.849500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:56.850023 (+ 523us) server_negotiation.cc:407] Connection header received
0810 01:02:56.850161 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:56.850165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:56.850227 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:56.850313 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:56.851766 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:56.852541 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:56.853912 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:56.854294 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:56.854456 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:56.856722 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:56.856726 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:56.856729 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:56.856760 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:56.859548 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:56.860163 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:56.860166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:56.860169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:56.860285 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:56.860627 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:56.860630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:56.860632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:56.860944 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:56.861122 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:56.861426 (+ 304us) server_negotiation.cc:299] Negotiation successful
0810 01:02:56.861595 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 01:02:56.866060 10302 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43606:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 01:02:56.873843 10302 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 01:02:56.903013 10852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:56.886922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:45543 (local address 127.0.0.1:46166)
0810 01:02:56.890809 (+ 3887us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:56.890859 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:56.891167 (+ 308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:56.892607 (+ 1440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:56.892619 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:56.892674 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:56.893220 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:56.893238 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:56.896293 (+ 3055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:56.896302 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:56.899519 (+ 3217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:56.899537 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:56.901893 (+ 2356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:56.901899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:56.902152 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:56.902215 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 01:02:56.902365 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 01:02:56.902594 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3627,"thread_start_us":9006,"threads_started":1}
I20260810 01:02:56.903085 10853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:56.887290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46166 (local address 127.20.81.129:45543)
0810 01:02:56.888138 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:56.888151 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:56.890972 (+ 2821us) server_negotiation.cc:407] Connection header received
0810 01:02:56.891533 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:56.891548 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:56.891908 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:56.892150 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:56.893493 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:56.895986 (+ 2493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:56.899703 (+ 3717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:56.901319 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:56.901476 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:56.901733 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:56.902486 (+ 753us) server_negotiation.cc:299] Negotiation successful
0810 01:02:56.902691 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":391,"threads_started":1}
I20260810 01:02:56.906536 10851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:56.887231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:38261 (local address 127.0.0.1:60326)
0810 01:02:56.896205 (+ 8974us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:56.896230 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:56.896379 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:56.897440 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:56.897446 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:56.897488 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:56.898081 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:56.898095 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:56.900143 (+ 2048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:56.900151 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:56.902288 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:56.902302 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:56.904328 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:56.904335 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:56.904526 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:56.904565 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 01:02:56.905953 (+ 1388us) client_negotiation.cc:240] Negotiation successful
0810 01:02:56.906130 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8833,"thread_start_us":3174,"threads_started":1}
I20260810 01:02:56.907416 10854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:56.887975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60326 (local address 127.20.81.131:38261)
0810 01:02:56.888975 (+ 1000us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:56.888986 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:56.896558 (+ 7572us) server_negotiation.cc:407] Connection header received
0810 01:02:56.896677 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:56.896686 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:56.896954 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:56.897154 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:56.898311 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:56.899999 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:56.902544 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:56.904108 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:56.904286 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:56.904546 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:56.906854 (+ 2308us) server_negotiation.cc:299] Negotiation successful
0810 01:02:56.907036 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":772,"thread_start_us":552,"threads_started":1}
I20260810 01:02:56.907806 10855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:56.890146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40062 (local address 127.20.81.130:42157)
0810 01:02:56.898188 (+ 8042us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:56.898198 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:56.898226 (+ 28us) server_negotiation.cc:407] Connection header received
0810 01:02:56.898332 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:56.898338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:56.898565 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:56.898758 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:56.899329 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:56.900842 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:56.903474 (+ 2632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:56.905276 (+ 1802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:56.905527 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:56.905860 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:56.906802 (+ 942us) server_negotiation.cc:299] Negotiation successful
0810 01:02:56.907124 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7871,"thread_start_us":7697,"threads_started":1}
I20260810 01:02:56.908596 10856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:56.890121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:42157 (local address 127.0.0.1:40062)
0810 01:02:56.896579 (+ 6458us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:56.896597 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:56.896702 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:56.898743 (+ 2041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:56.898750 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:56.898775 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:56.899172 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:56.899181 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:56.901051 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:56.901056 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:56.903289 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:56.903303 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:56.906115 (+ 2812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:56.906125 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:56.906452 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:56.906533 (+ 81us) client_negotiation.cc:769] Sending connection context
0810 01:02:56.907629 (+ 1096us) client_negotiation.cc:240] Negotiation successful
0810 01:02:56.908146 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6339,"mutex_wait_us":1,"thread_start_us":6663,"threads_started":1}
I20260810 01:02:56.908453 10441 tablet_service.cc:1467] Processing CreateTablet for tablet 9f7d94c42d73458b81cc65d82d54b55e (DEFAULT_TABLE table=test-table [id=19ad762e07fd45118e12bd14d320683b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:02:56.912758 10760 tablet_service.cc:1467] Processing CreateTablet for tablet 9f7d94c42d73458b81cc65d82d54b55e (DEFAULT_TABLE table=test-table [id=19ad762e07fd45118e12bd14d320683b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:02:56.913441 10603 tablet_service.cc:1467] Processing CreateTablet for tablet 9f7d94c42d73458b81cc65d82d54b55e (DEFAULT_TABLE table=test-table [id=19ad762e07fd45118e12bd14d320683b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:02:56.914142 10760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f7d94c42d73458b81cc65d82d54b55e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:02:56.914912 10603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f7d94c42d73458b81cc65d82d54b55e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:02:56.918896 10441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f7d94c42d73458b81cc65d82d54b55e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:02:56.928355 10857 tablet_bootstrap.cc:492] T 9f7d94c42d73458b81cc65d82d54b55e P 674bdeedd3934b77b4b34b5ca949db8b: Bootstrap starting.
I20260810 01:02:56.932605 10857 tablet_bootstrap.cc:654] T 9f7d94c42d73458b81cc65d82d54b55e P 674bdeedd3934b77b4b34b5ca949db8b: Neither blocks nor log segments found. Creating new log.
I20260810 01:02:56.934382 10857 log.cc:824] T 9f7d94c42d73458b81cc65d82d54b55e P 674bdeedd3934b77b4b34b5ca949db8b: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:56.943578 10858 tablet_bootstrap.cc:492] T 9f7d94c42d73458b81cc65d82d54b55e P 034384f6897546358c09883c8053cd2c: Bootstrap starting.
I20260810 01:02:56.945019 10857 tablet_bootstrap.cc:492] T 9f7d94c42d73458b81cc65d82d54b55e P 674bdeedd3934b77b4b34b5ca949db8b: No bootstrap required, opened a new log
I20260810 01:02:56.945037 10860 tablet_bootstrap.cc:492] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585: Bootstrap starting.
I20260810 01:02:56.945432 10857 ts_tablet_manager.cc:1397] T 9f7d94c42d73458b81cc65d82d54b55e P 674bdeedd3934b77b4b34b5ca949db8b: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.005s
I20260810 01:02:56.947835 10858 tablet_bootstrap.cc:654] T 9f7d94c42d73458b81cc65d82d54b55e P 034384f6897546358c09883c8053cd2c: Neither blocks nor log segments found. Creating new log.
I20260810 01:02:56.948591 10860 tablet_bootstrap.cc:654] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585: Neither blocks nor log segments found. Creating new log.
I20260810 01:02:56.949398 10858 log.cc:824] T 9f7d94c42d73458b81cc65d82d54b55e P 034384f6897546358c09883c8053cd2c: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:56.950191 10860 log.cc:824] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:56.960425 10857 raft_consensus.cc:348] T 9f7d94c42d73458b81cc65d82d54b55e P 674bdeedd3934b77b4b34b5ca949db8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95ea9e4492c84655906846f862cd0585" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45543 } } peers { permanent_uuid: "674bdeedd3934b77b4b34b5ca949db8b" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 38261 } } peers { permanent_uuid: "034384f6897546358c09883c8053cd2c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 42157 } }
I20260810 01:02:56.961774 10857 raft_consensus.cc:374] T 9f7d94c42d73458b81cc65d82d54b55e P 674bdeedd3934b77b4b34b5ca949db8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:02:56.961931 10857 raft_consensus.cc:729] T 9f7d94c42d73458b81cc65d82d54b55e P 674bdeedd3934b77b4b34b5ca949db8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 674bdeedd3934b77b4b34b5ca949db8b, State: Initialized, Role: FOLLOWER
I20260810 01:02:56.962692 10857 consensus_queue.cc:260] T 9f7d94c42d73458b81cc65d82d54b55e P 674bdeedd3934b77b4b34b5ca949db8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95ea9e4492c84655906846f862cd0585" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45543 } } peers { permanent_uuid: "674bdeedd3934b77b4b34b5ca949db8b" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 38261 } } peers { permanent_uuid: "034384f6897546358c09883c8053cd2c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 42157 } }
I20260810 01:02:56.963941 10831 heartbeater.cc:503] Master 127.20.81.190:43537 was elected leader, sending a full tablet report...
I20260810 01:02:56.964361 10860 tablet_bootstrap.cc:492] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585: No bootstrap required, opened a new log
I20260810 01:02:56.964375 10858 tablet_bootstrap.cc:492] T 9f7d94c42d73458b81cc65d82d54b55e P 034384f6897546358c09883c8053cd2c: No bootstrap required, opened a new log
I20260810 01:02:56.964618 10857 ts_tablet_manager.cc:1428] T 9f7d94c42d73458b81cc65d82d54b55e P 674bdeedd3934b77b4b34b5ca949db8b: Time spent starting tablet: real 0.019s user 0.013s sys 0.003s
I20260810 01:02:56.964636 10860 ts_tablet_manager.cc:1397] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.016s
I20260810 01:02:56.964673 10858 ts_tablet_manager.cc:1397] T 9f7d94c42d73458b81cc65d82d54b55e P 034384f6897546358c09883c8053cd2c: Time spent bootstrapping tablet: real 0.021s user 0.009s sys 0.009s
I20260810 01:02:56.975243 10858 raft_consensus.cc:348] T 9f7d94c42d73458b81cc65d82d54b55e P 034384f6897546358c09883c8053cd2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95ea9e4492c84655906846f862cd0585" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45543 } } peers { permanent_uuid: "674bdeedd3934b77b4b34b5ca949db8b" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 38261 } } peers { permanent_uuid: "034384f6897546358c09883c8053cd2c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 42157 } }
I20260810 01:02:56.975245 10860 raft_consensus.cc:348] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95ea9e4492c84655906846f862cd0585" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45543 } } peers { permanent_uuid: "674bdeedd3934b77b4b34b5ca949db8b" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 38261 } } peers { permanent_uuid: "034384f6897546358c09883c8053cd2c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 42157 } }
I20260810 01:02:56.975692 10860 raft_consensus.cc:374] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:02:56.975742 10858 raft_consensus.cc:374] T 9f7d94c42d73458b81cc65d82d54b55e P 034384f6897546358c09883c8053cd2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:02:56.975842 10860 raft_consensus.cc:729] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95ea9e4492c84655906846f862cd0585, State: Initialized, Role: FOLLOWER
I20260810 01:02:56.975872 10858 raft_consensus.cc:729] T 9f7d94c42d73458b81cc65d82d54b55e P 034384f6897546358c09883c8053cd2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 034384f6897546358c09883c8053cd2c, State: Initialized, Role: FOLLOWER
I20260810 01:02:56.976557 10860 consensus_queue.cc:260] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95ea9e4492c84655906846f862cd0585" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45543 } } peers { permanent_uuid: "674bdeedd3934b77b4b34b5ca949db8b" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 38261 } } peers { permanent_uuid: "034384f6897546358c09883c8053cd2c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 42157 } }
I20260810 01:02:56.976560 10858 consensus_queue.cc:260] T 9f7d94c42d73458b81cc65d82d54b55e P 034384f6897546358c09883c8053cd2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95ea9e4492c84655906846f862cd0585" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45543 } } peers { permanent_uuid: "674bdeedd3934b77b4b34b5ca949db8b" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 38261 } } peers { permanent_uuid: "034384f6897546358c09883c8053cd2c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 42157 } }
I20260810 01:02:56.977815 10678 heartbeater.cc:503] Master 127.20.81.190:43537 was elected leader, sending a full tablet report...
I20260810 01:02:56.979828 10860 ts_tablet_manager.cc:1428] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20260810 01:02:56.982854 10858 ts_tablet_manager.cc:1428] T 9f7d94c42d73458b81cc65d82d54b55e P 034384f6897546358c09883c8053cd2c: Time spent starting tablet: real 0.018s user 0.012s sys 0.003s
W20260810 01:02:56.996275 10679 tablet.cc:2369] T 9f7d94c42d73458b81cc65d82d54b55e P 034384f6897546358c09883c8053cd2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 01:02:57.065196 10833 tablet.cc:2369] T 9f7d94c42d73458b81cc65d82d54b55e P 674bdeedd3934b77b4b34b5ca949db8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 01:02:57.192970 10518 tablet.cc:2369] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:02:57.286573 10867 raft_consensus.cc:482] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:02:57.286931 10867 raft_consensus.cc:504] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95ea9e4492c84655906846f862cd0585" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45543 } } peers { permanent_uuid: "674bdeedd3934b77b4b34b5ca949db8b" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 38261 } } peers { permanent_uuid: "034384f6897546358c09883c8053cd2c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 42157 } }
I20260810 01:02:57.288820 10867 leader_election.cc:288] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 674bdeedd3934b77b4b34b5ca949db8b (127.20.81.131:38261), 034384f6897546358c09883c8053cd2c (127.20.81.130:42157)
I20260810 01:02:57.295823 10855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:57.289268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:48271 (local address 127.20.81.130:42157)
0810 01:02:57.289559 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:57.289569 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:57.290360 (+ 791us) server_negotiation.cc:407] Connection header received
0810 01:02:57.290624 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:57.290628 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:57.290689 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:57.290787 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:57.291430 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:57.292146 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:57.293989 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:57.294840 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:57.295031 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:57.295114 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:57.295415 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 01:02:57.295550 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 01:02:57.295857 10854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:57.289401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:36775 (local address 127.20.81.131:38261)
0810 01:02:57.289642 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:57.289651 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:57.290265 (+ 614us) server_negotiation.cc:407] Connection header received
0810 01:02:57.290521 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:57.290525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:57.290588 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:57.290689 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:02:57.291268 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:57.292004 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:57.293991 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:57.294851 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:57.295035 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:57.295126 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:57.295448 (+ 322us) server_negotiation.cc:299] Negotiation successful
0810 01:02:57.295593 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 01:02:57.296015 10868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:57.289189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:42157 (local address 127.20.81.129:48271)
0810 01:02:57.290243 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:57.290257 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:57.290467 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:57.290805 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:57.290809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:57.290848 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:57.291264 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:57.291273 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:57.292289 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:57.292294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:57.293821 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:57.293833 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:57.295087 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:57.295093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:57.295247 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:57.295270 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 01:02:57.295387 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 01:02:57.295550 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":943,"mutex_wait_us":11,"thread_start_us":583,"threads_started":1}
I20260810 01:02:57.296072 10869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:57.289021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:38261 (local address 127.20.81.129:36775)
0810 01:02:57.290146 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:57.290163 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:57.290336 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:57.290707 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:57.290711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:57.290734 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:02:57.291110 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:57.291118 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:57.292142 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:57.292148 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:57.293828 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:57.293840 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:57.295144 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:57.295152 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:57.295289 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:57.295311 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 01:02:57.295419 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 01:02:57.295592 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"mutex_wait_us":134,"thread_start_us":872,"threads_started":1}
I20260810 01:02:57.297145 10783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f7d94c42d73458b81cc65d82d54b55e" candidate_uuid: "95ea9e4492c84655906846f862cd0585" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "674bdeedd3934b77b4b34b5ca949db8b" is_pre_election: true
I20260810 01:02:57.297189 10629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f7d94c42d73458b81cc65d82d54b55e" candidate_uuid: "95ea9e4492c84655906846f862cd0585" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "034384f6897546358c09883c8053cd2c" is_pre_election: true
I20260810 01:02:57.297662 10783 raft_consensus.cc:2454] T 9f7d94c42d73458b81cc65d82d54b55e P 674bdeedd3934b77b4b34b5ca949db8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95ea9e4492c84655906846f862cd0585 in term 0.
I20260810 01:02:57.297683 10629 raft_consensus.cc:2454] T 9f7d94c42d73458b81cc65d82d54b55e P 034384f6897546358c09883c8053cd2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95ea9e4492c84655906846f862cd0585 in term 0.
I20260810 01:02:57.298448 10392 leader_election.cc:302] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 034384f6897546358c09883c8053cd2c, 95ea9e4492c84655906846f862cd0585; no voters:
I20260810 01:02:57.298898 10867 raft_consensus.cc:2781] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:02:57.299018 10867 raft_consensus.cc:482] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:02:57.299154 10867 raft_consensus.cc:3037] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:02:57.301452 10867 raft_consensus.cc:504] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95ea9e4492c84655906846f862cd0585" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45543 } } peers { permanent_uuid: "674bdeedd3934b77b4b34b5ca949db8b" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 38261 } } peers { permanent_uuid: "034384f6897546358c09883c8053cd2c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 42157 } }
I20260810 01:02:57.302137 10867 leader_election.cc:288] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585 [CANDIDATE]: Term 1 election: Requested vote from peers 674bdeedd3934b77b4b34b5ca949db8b (127.20.81.131:38261), 034384f6897546358c09883c8053cd2c (127.20.81.130:42157)
I20260810 01:02:57.302591 10783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f7d94c42d73458b81cc65d82d54b55e" candidate_uuid: "95ea9e4492c84655906846f862cd0585" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "674bdeedd3934b77b4b34b5ca949db8b"
I20260810 01:02:57.302788 10783 raft_consensus.cc:3037] T 9f7d94c42d73458b81cc65d82d54b55e P 674bdeedd3934b77b4b34b5ca949db8b [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:02:57.302884 10629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f7d94c42d73458b81cc65d82d54b55e" candidate_uuid: "95ea9e4492c84655906846f862cd0585" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "034384f6897546358c09883c8053cd2c"
I20260810 01:02:57.303143 10629 raft_consensus.cc:3037] T 9f7d94c42d73458b81cc65d82d54b55e P 034384f6897546358c09883c8053cd2c [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:02:57.305094 10783 raft_consensus.cc:2454] T 9f7d94c42d73458b81cc65d82d54b55e P 674bdeedd3934b77b4b34b5ca949db8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95ea9e4492c84655906846f862cd0585 in term 1.
I20260810 01:02:57.305564 10393 leader_election.cc:302] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 674bdeedd3934b77b4b34b5ca949db8b, 95ea9e4492c84655906846f862cd0585; no voters:
I20260810 01:02:57.305943 10867 raft_consensus.cc:2781] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:02:57.305979 10629 raft_consensus.cc:2454] T 9f7d94c42d73458b81cc65d82d54b55e P 034384f6897546358c09883c8053cd2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95ea9e4492c84655906846f862cd0585 in term 1.
I20260810 01:02:57.306617 10867 raft_consensus.cc:686] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585 [term 1 LEADER]: Becoming Leader. State: Replica: 95ea9e4492c84655906846f862cd0585, State: Running, Role: LEADER
I20260810 01:02:57.307108 10867 consensus_queue.cc:237] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95ea9e4492c84655906846f862cd0585" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45543 } } peers { permanent_uuid: "674bdeedd3934b77b4b34b5ca949db8b" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 38261 } } peers { permanent_uuid: "034384f6897546358c09883c8053cd2c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 42157 } }
I20260810 01:02:57.312319 10302 catalog_manager.cc:5495] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95ea9e4492c84655906846f862cd0585 (127.20.81.129). New cstate: current_term: 1 leader_uuid: "95ea9e4492c84655906846f862cd0585" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95ea9e4492c84655906846f862cd0585" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 45543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "674bdeedd3934b77b4b34b5ca949db8b" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "034384f6897546358c09883c8053cd2c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 42157 } health_report { overall_health: UNKNOWN } } }
I20260810 01:02:57.382700 10853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:57.376689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46180 (local address 127.20.81.129:45543)
0810 01:02:57.376912 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:57.376918 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:57.377299 (+ 381us) server_negotiation.cc:407] Connection header received
0810 01:02:57.377502 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:57.377506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:57.377597 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:57.377747 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:02:57.378148 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:57.379217 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:57.380667 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:57.381074 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:57.381177 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:57.381623 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:02:57.382158 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:02:57.382271 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:57.382353 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 01:02:57.382495 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 01:02:57.391413 10783 raft_consensus.cc:1261] T 9f7d94c42d73458b81cc65d82d54b55e P 674bdeedd3934b77b4b34b5ca949db8b [term 1 FOLLOWER]: Refusing update from remote peer 95ea9e4492c84655906846f862cd0585: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:02:57.391410 10629 raft_consensus.cc:1261] T 9f7d94c42d73458b81cc65d82d54b55e P 034384f6897546358c09883c8053cd2c [term 1 FOLLOWER]: Refusing update from remote peer 95ea9e4492c84655906846f862cd0585: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:02:57.392372 10870 consensus_queue.cc:1035] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585 [LEADER]: Connected to new peer: Peer: permanent_uuid: "034384f6897546358c09883c8053cd2c" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 42157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:02:57.392706 10867 consensus_queue.cc:1035] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585 [LEADER]: Connected to new peer: Peer: permanent_uuid: "674bdeedd3934b77b4b34b5ca949db8b" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:02:57.423082 10300 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:02:57.423381 10300 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:02:57.426791 10300 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bf8f48e8cbcc412f87ac325c278cf559: Sending DeleteTablet for 3 replicas of tablet 9f7d94c42d73458b81cc65d82d54b55e
I20260810 01:02:57.427887 10441 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f7d94c42d73458b81cc65d82d54b55e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:02:57 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:46166
I20260810 01:02:57.427980 10760 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f7d94c42d73458b81cc65d82d54b55e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:02:57 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:60326
I20260810 01:02:57.428078 10603 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f7d94c42d73458b81cc65d82d54b55e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:02:57 UTC) from {username='kudu', principal='kudu/127.20.81.190@KRBTEST.COM'} at 127.0.0.1:40062
I20260810 01:02:57.432127 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 10370
I20260810 01:02:57.438875 10889 tablet_replica.cc:330] T 9f7d94c42d73458b81cc65d82d54b55e P 034384f6897546358c09883c8053cd2c: stopping tablet replica
I20260810 01:02:57.440253 10889 raft_consensus.cc:2229] T 9f7d94c42d73458b81cc65d82d54b55e P 034384f6897546358c09883c8053cd2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:02:57.441941 10889 raft_consensus.cc:2258] T 9f7d94c42d73458b81cc65d82d54b55e P 034384f6897546358c09883c8053cd2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:02:57.446228 10888 tablet_replica.cc:330] T 9f7d94c42d73458b81cc65d82d54b55e P 674bdeedd3934b77b4b34b5ca949db8b: stopping tablet replica
I20260810 01:02:57.447258 10888 raft_consensus.cc:2229] T 9f7d94c42d73458b81cc65d82d54b55e P 674bdeedd3934b77b4b34b5ca949db8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:02:57.448419 10888 raft_consensus.cc:2258] T 9f7d94c42d73458b81cc65d82d54b55e P 674bdeedd3934b77b4b34b5ca949db8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:02:57.448671 10889 ts_tablet_manager.cc:1905] T 9f7d94c42d73458b81cc65d82d54b55e P 034384f6897546358c09883c8053cd2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:02:57.451344 10888 ts_tablet_manager.cc:1905] T 9f7d94c42d73458b81cc65d82d54b55e P 674bdeedd3934b77b4b34b5ca949db8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:02:57.453258 10887 tablet_replica.cc:330] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585: stopping tablet replica
I20260810 01:02:57.453681 10887 raft_consensus.cc:2229] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585 [term 1 LEADER]: Raft consensus shutting down.
I20260810 01:02:57.454365 10887 raft_consensus.cc:2258] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:02:57.456362 10889 ts_tablet_manager.cc:1918] T 9f7d94c42d73458b81cc65d82d54b55e P 034384f6897546358c09883c8053cd2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:02:57.456521 10889 log.cc:1196] T 9f7d94c42d73458b81cc65d82d54b55e P 034384f6897546358c09883c8053cd2c: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/9f7d94c42d73458b81cc65d82d54b55e
I20260810 01:02:57.457162 10889 ts_tablet_manager.cc:1939] T 9f7d94c42d73458b81cc65d82d54b55e P 034384f6897546358c09883c8053cd2c: Deleting consensus metadata
I20260810 01:02:57.458629 10887 ts_tablet_manager.cc:1905] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:02:57.459182 10285 catalog_manager.cc:4841] TS 034384f6897546358c09883c8053cd2c (127.20.81.130:42157): tablet 9f7d94c42d73458b81cc65d82d54b55e (table test-table [id=19ad762e07fd45118e12bd14d320683b]) successfully deleted
I20260810 01:02:57.459609 10888 ts_tablet_manager.cc:1918] T 9f7d94c42d73458b81cc65d82d54b55e P 674bdeedd3934b77b4b34b5ca949db8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:02:57.459812 10888 log.cc:1196] T 9f7d94c42d73458b81cc65d82d54b55e P 674bdeedd3934b77b4b34b5ca949db8b: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/9f7d94c42d73458b81cc65d82d54b55e
I20260810 01:02:57.460419 10888 ts_tablet_manager.cc:1939] T 9f7d94c42d73458b81cc65d82d54b55e P 674bdeedd3934b77b4b34b5ca949db8b: Deleting consensus metadata
I20260810 01:02:57.462208 10286 catalog_manager.cc:4841] TS 674bdeedd3934b77b4b34b5ca949db8b (127.20.81.131:38261): tablet 9f7d94c42d73458b81cc65d82d54b55e (table test-table [id=19ad762e07fd45118e12bd14d320683b]) successfully deleted
Aug 10 01:02:57 dist-test-slave-gp6n krb5kdc[10251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.129@KRBTEST.COM
I20260810 01:02:57.467118 10887 ts_tablet_manager.cc:1918] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:02:57.467319 10887 log.cc:1196] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323445691443-20806-0/minicluster-data/ts-0/wal/wals/9f7d94c42d73458b81cc65d82d54b55e
I20260810 01:02:57.467839 10887 ts_tablet_manager.cc:1939] T 9f7d94c42d73458b81cc65d82d54b55e P 95ea9e4492c84655906846f862cd0585: Deleting consensus metadata
I20260810 01:02:57.469375 10284 catalog_manager.cc:4841] TS 95ea9e4492c84655906846f862cd0585 (127.20.81.129:45543): tablet 9f7d94c42d73458b81cc65d82d54b55e (table test-table [id=19ad762e07fd45118e12bd14d320683b]) successfully deleted
I20260810 01:02:57.474248 10853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:57.433953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46194 (local address 127.20.81.129:45543)
0810 01:02:57.434185 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:57.434194 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:57.446312 (+ 12118us) server_negotiation.cc:407] Connection header received
0810 01:02:57.446639 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:57.446649 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:57.446774 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:57.446867 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:57.448997 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:57.450337 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:57.452603 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:57.453400 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:57.453534 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:57.466799 (+ 13265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:57.466814 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:57.466831 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:57.466879 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:57.471300 (+ 4421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:57.472024 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:57.472034 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:57.472037 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:57.472246 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:57.472630 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:57.472636 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:57.472639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:57.473100 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:57.473417 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:57.473619 (+ 202us) server_negotiation.cc:299] Negotiation successful
0810 01:02:57.473876 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 01:02:57.475116 10511 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:57.575071 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 10370
I20260810 01:02:57.597980 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 10524
Aug 10 01:02:57 dist-test-slave-gp6n krb5kdc[10251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.130@KRBTEST.COM
I20260810 01:02:57.613215 10855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:57.598827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40070 (local address 127.20.81.130:42157)
0810 01:02:57.598996 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:57.599001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:57.599021 (+ 20us) server_negotiation.cc:407] Connection header received
0810 01:02:57.599179 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:57.599183 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:57.599244 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:57.599339 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:57.600575 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:57.601269 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:57.602588 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:57.603024 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:57.603121 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:57.607272 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:57.607286 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:57.607301 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:57.607333 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:57.610255 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:57.610997 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:57.611005 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:57.611007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:57.611143 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:57.611500 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:57.611504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:57.611506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:57.611897 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:57.612145 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:57.612581 (+ 436us) server_negotiation.cc:299] Negotiation successful
0810 01:02:57.612835 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 01:02:57.614099 10673 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:57.707455 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 10524
I20260810 01:02:57.728871 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 10686
Aug 10 01:02:57 dist-test-slave-gp6n krb5kdc[10251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.131@KRBTEST.COM
I20260810 01:02:57.744772 10854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:57.729674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60330 (local address 127.20.81.131:38261)
0810 01:02:57.729862 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:57.729867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:57.729886 (+ 19us) server_negotiation.cc:407] Connection header received
0810 01:02:57.729976 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:57.729980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:57.730044 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:57.730163 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:57.731589 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:57.732324 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:57.733561 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:57.733953 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:57.734046 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:57.738792 (+ 4746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:57.738874 (+ 82us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:57.738897 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:02:57.738952 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:57.742010 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:57.742778 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:57.742787 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:57.742790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:57.742963 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:02:57.743383 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:02:57.743388 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:02:57.743391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:02:57.743757 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:57.743991 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:57.744321 (+ 330us) server_negotiation.cc:299] Negotiation successful
0810 01:02:57.744493 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 01:02:57.745607 10825 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:57.835186 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 10686
I20260810 01:02:57.858211 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 10264
I20260810 01:02:57.859557 10334 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:02:57.971784 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 10264
2026-08-10T01:02:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5008 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T01:02:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:02:58Z Disabled control of system clock
I20260810 01:02:58.069474 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:40125
--webserver_interface=127.20.81.190
--webserver_port=0
--builtin_ntp_servers=127.20.81.148:39553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:40125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 01:02:58.423368 10900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:58.423650 10900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:58.423815 10900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:58.431193 10900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:02:58.431298 10900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:58.431387 10900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:58.431484 10900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:02:58.431556 10900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:02:58.440972 10900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:39553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:40125
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:40125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:58.442299 10900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:58.444218 10900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:58.456301 10906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:58.456248 10905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:58.456323 10908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:58.458197 10900 server_base.cc:1034] running on GCE node
I20260810 01:02:58.459208 10900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:58.460845 10900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:58.462078 10900 hybrid_clock.cc:648] HybridClock initialized: now 1786323778462007 us; error 59 us; skew 500 ppm
I20260810 01:02:58.462649 10900 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:02:58.465339 10900 webserver.cc:460] Webserver started at http://127.20.81.190:42501/ using document root <none> and password file <none>
I20260810 01:02:58.466096 10900 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:58.466216 10900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:58.466636 10900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:58.469619 10900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "9374ce40127b48d5951b53a3fa21569d"
format_stamp: "Formatted at 2026-08-10 01:02:58 on dist-test-slave-gp6n"
server_key: "ad5ea9e26fc80d5447e2145d1d49d7f1"
server_key_iv: "5d485d96f3ccfecf75ee4197d54970c0"
server_key_version: "encryptionkey@0"
I20260810 01:02:58.470605 10900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "9374ce40127b48d5951b53a3fa21569d"
format_stamp: "Formatted at 2026-08-10 01:02:58 on dist-test-slave-gp6n"
server_key: "ad5ea9e26fc80d5447e2145d1d49d7f1"
server_key_iv: "5d485d96f3ccfecf75ee4197d54970c0"
server_key_version: "encryptionkey@0"
I20260810 01:02:58.476017 10900 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 01:02:58.479604 10914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:58.481024 10900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 01:02:58.481211 10900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "9374ce40127b48d5951b53a3fa21569d"
format_stamp: "Formatted at 2026-08-10 01:02:58 on dist-test-slave-gp6n"
server_key: "ad5ea9e26fc80d5447e2145d1d49d7f1"
server_key_iv: "5d485d96f3ccfecf75ee4197d54970c0"
server_key_version: "encryptionkey@0"
I20260810 01:02:58.481434 10900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:58.507717 10900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:58.508641 10900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:58.508958 10900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:58.529065 10900 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:40125
I20260810 01:02:58.529156 10971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:40125 every 8 connection(s)
I20260810 01:02:58.530475 10900 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:02:58.534730 10972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:02:58.539108 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 10900
I20260810 01:02:58.539295 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:02:58.539753 20806 external_mini_cluster.cc:1428] Setting key 877483c845e2277e6dc83e773763fddb
I20260810 01:02:58.548097 10972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d: Bootstrap starting.
I20260810 01:02:58.551409 10972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d: Neither blocks nor log segments found. Creating new log.
I20260810 01:02:58.552501 10972 log.cc:824] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d: Log is configured to *not* fsync() on all Append() calls
I20260810 01:02:58.554011 10976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:58.541744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37696 (local address 127.20.81.190:40125)
0810 01:02:58.542880 (+ 1136us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:58.542891 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:58.542933 (+ 42us) server_negotiation.cc:407] Connection header received
0810 01:02:58.544619 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:58.544642 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:58.545138 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:58.545568 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:58.546209 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:58.547562 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:58.548978 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:58.549704 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:58.549906 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:58.551183 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:58.551190 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:58.551204 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:02:58.551881 (+ 677us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:58.551972 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:02:58.551993 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:58.552126 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:58.552356 (+ 230us) server_negotiation.cc:299] Negotiation successful
0810 01:02:58.552707 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":411,"threads_started":1}
I20260810 01:02:58.555543 10972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d: No bootstrap required, opened a new log
I20260810 01:02:58.563392 10972 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9374ce40127b48d5951b53a3fa21569d" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40125 } }
I20260810 01:02:58.563758 10972 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:02:58.563884 10972 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9374ce40127b48d5951b53a3fa21569d, State: Initialized, Role: FOLLOWER
I20260810 01:02:58.564507 10972 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9374ce40127b48d5951b53a3fa21569d" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40125 } }
I20260810 01:02:58.564746 10972 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:02:58.564878 10972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:02:58.565065 10972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:02:58.567322 10972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9374ce40127b48d5951b53a3fa21569d" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40125 } }
I20260810 01:02:58.567847 10972 leader_election.cc:302] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9374ce40127b48d5951b53a3fa21569d; no voters:
I20260810 01:02:58.568475 10972 leader_election.cc:288] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:02:58.568630 10979 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:02:58.569418 10979 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d [term 1 LEADER]: Becoming Leader. State: Replica: 9374ce40127b48d5951b53a3fa21569d, State: Running, Role: LEADER
I20260810 01:02:58.570045 10979 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9374ce40127b48d5951b53a3fa21569d" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40125 } }
I20260810 01:02:58.570849 10972 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:02:58.573076 10980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9374ce40127b48d5951b53a3fa21569d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9374ce40127b48d5951b53a3fa21569d" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40125 } } }
I20260810 01:02:58.573761 10980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d [sys.catalog]: This master's current role is: LEADER
I20260810 01:02:58.573549 10982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9374ce40127b48d5951b53a3fa21569d. Latest consensus state: current_term: 1 leader_uuid: "9374ce40127b48d5951b53a3fa21569d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9374ce40127b48d5951b53a3fa21569d" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 40125 } } }
I20260810 01:02:58.573957 10982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d [sys.catalog]: This master's current role is: LEADER
I20260810 01:02:58.575389 10988 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:02:58.580492 10988 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:02:58.589643 10988 catalog_manager.cc:1347] Generated new cluster ID: aff338c3b8234369b64340379d5ef529
I20260810 01:02:58.589745 10988 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:02:58.608513 10988 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:02:58.609934 10988 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:02:58.625059 10988 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d: Generated new TSK 0
I20260810 01:02:58.626035 10988 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:02:58.637923 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.129:0
--local_ip_for_outbound_sockets=127.20.81.129
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:40125
--builtin_ntp_servers=127.20.81.148:39553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 01:02:58.992075 11001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:58.992380 11001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:58.992517 11001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:59.000020 11001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:59.000176 11001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:59.000391 11001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.129
I20260810 01:02:59.009802 11001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:39553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.81.129
--webserver_port=0
--tserver_master_addrs=127.20.81.190:40125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.129
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:59.011289 11001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:59.013146 11001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:59.026679 11008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:59.026785 11011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:59.026727 11007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:59.027578 11001 server_base.cc:1034] running on GCE node
I20260810 01:02:59.028185 11001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:59.029212 11001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:59.030501 11001 hybrid_clock.cc:648] HybridClock initialized: now 1786323779030423 us; error 57 us; skew 500 ppm
I20260810 01:02:59.030964 11001 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:02:59.033987 11001 webserver.cc:460] Webserver started at http://127.20.81.129:34097/ using document root <none> and password file <none>
I20260810 01:02:59.034981 11001 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:59.035120 11001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:59.035538 11001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:59.039247 11001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-0/data/instance:
uuid: "c75d08fa4f5d433691dd2aa7d9aae9c6"
format_stamp: "Formatted at 2026-08-10 01:02:59 on dist-test-slave-gp6n"
server_key: "7fe3c3ba04a58194a5ee4cbc0657b1d0"
server_key_iv: "4c30ddd54c5069107c840e9a4516af0a"
server_key_version: "encryptionkey@0"
I20260810 01:02:59.040045 11001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance:
uuid: "c75d08fa4f5d433691dd2aa7d9aae9c6"
format_stamp: "Formatted at 2026-08-10 01:02:59 on dist-test-slave-gp6n"
server_key: "7fe3c3ba04a58194a5ee4cbc0657b1d0"
server_key_iv: "4c30ddd54c5069107c840e9a4516af0a"
server_key_version: "encryptionkey@0"
I20260810 01:02:59.045486 11001 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 01:02:59.049074 11018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:59.050720 11001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:02:59.050935 11001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-0/wal
uuid: "c75d08fa4f5d433691dd2aa7d9aae9c6"
format_stamp: "Formatted at 2026-08-10 01:02:59 on dist-test-slave-gp6n"
server_key: "7fe3c3ba04a58194a5ee4cbc0657b1d0"
server_key_iv: "4c30ddd54c5069107c840e9a4516af0a"
server_key_version: "encryptionkey@0"
I20260810 01:02:59.051149 11001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:59.068907 11001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:59.069772 11001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:59.070099 11001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:59.071281 11001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:02:59.072820 11001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:02:59.072943 11001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:59.073052 11001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:02:59.073148 11001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:59.106698 11001 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.129:40055
I20260810 01:02:59.106755 11142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.129:40055 every 8 connection(s)
I20260810 01:02:59.108144 11001 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-0/data/info.pb
I20260810 01:02:59.112555 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 11001
I20260810 01:02:59.112753 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-0/wal/instance
I20260810 01:02:59.113332 20806 external_mini_cluster.cc:1428] Setting key 55c9e9902e8fabbe8fc466962c7d9bfa
I20260810 01:02:59.116756 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.130:0
--local_ip_for_outbound_sockets=127.20.81.130
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:40125
--builtin_ntp_servers=127.20.81.148:39553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 01:02:59.124281 11149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:59.110915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.129:51147 (local address 127.20.81.190:40125)
0810 01:02:59.111517 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:59.111526 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:59.113356 (+ 1830us) server_negotiation.cc:407] Connection header received
0810 01:02:59.115755 (+ 2399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:59.115759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:59.115830 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:59.115920 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:59.117456 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:59.118205 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:59.120856 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:59.121610 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:59.121756 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:59.123192 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:59.123201 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:59.123208 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:02:59.123237 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:59.123294 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:02:59.123313 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:59.123412 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:59.123667 (+ 255us) server_negotiation.cc:299] Negotiation successful
0810 01:02:59.123787 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":266,"threads_started":1}
I20260810 01:02:59.125464 11150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:59.111602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:40125 (local address 127.20.81.129:51147)
0810 01:02:59.113147 (+ 1545us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:59.113206 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:59.115445 (+ 2239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:59.116225 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:59.116237 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:59.116364 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:02:59.117256 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:59.117286 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:59.118376 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:59.118384 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:59.120631 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:59.120648 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:59.121841 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:59.121846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:59.122070 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:59.122854 (+ 784us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 01:02:59.122857 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:02:59.122900 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 01:02:59.122989 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:02:59.123415 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:02:59.123423 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:02:59.123430 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 01:02:59.123606 (+ 176us) client_negotiation.cc:240] Negotiation successful
0810 01:02:59.124065 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":932,"thread_start_us":657,"threads_started":1}
I20260810 01:02:59.127199 11145 heartbeater.cc:347] Connected to a master server at 127.20.81.190:40125
I20260810 01:02:59.127542 11145 heartbeater.cc:464] Registering TS with master...
I20260810 01:02:59.128541 11145 heartbeater.cc:511] Master 127.20.81.190:40125 requested a full tablet report, sending...
I20260810 01:02:59.130976 10936 ts_manager.cc:194] Registered new tserver with Master: c75d08fa4f5d433691dd2aa7d9aae9c6 (127.20.81.129:40055)
I20260810 01:02:59.132473 10936 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.81.129:51147
W20260810 01:02:59.471367 11151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:59.471704 11151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:59.471863 11151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:59.479329 11151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:59.479461 11151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:59.479688 11151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.130
I20260810 01:02:59.489169 11151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:39553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.81.130
--webserver_port=0
--tserver_master_addrs=127.20.81.190:40125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.130
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:59.490572 11151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:59.492354 11151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:59.505434 11159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:59.505514 11161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:59.505481 11156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:59.506510 11151 server_base.cc:1034] running on GCE node
I20260810 01:02:59.507134 11151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:59.508090 11151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:59.509343 11151 hybrid_clock.cc:648] HybridClock initialized: now 1786323779509286 us; error 56 us; skew 500 ppm
I20260810 01:02:59.509689 11151 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:02:59.512068 11151 webserver.cc:460] Webserver started at http://127.20.81.130:40839/ using document root <none> and password file <none>
I20260810 01:02:59.512773 11151 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:59.512923 11151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:59.513311 11151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:59.516291 11151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-1/data/instance:
uuid: "f80e3559b46a4a7c89b810e26038b39e"
format_stamp: "Formatted at 2026-08-10 01:02:59 on dist-test-slave-gp6n"
server_key: "83ab807f013b3ba593060f283e72db4b"
server_key_iv: "3d5b515171d4f63565feed9ded28a173"
server_key_version: "encryptionkey@0"
I20260810 01:02:59.517076 11151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance:
uuid: "f80e3559b46a4a7c89b810e26038b39e"
format_stamp: "Formatted at 2026-08-10 01:02:59 on dist-test-slave-gp6n"
server_key: "83ab807f013b3ba593060f283e72db4b"
server_key_iv: "3d5b515171d4f63565feed9ded28a173"
server_key_version: "encryptionkey@0"
I20260810 01:02:59.522339 11151 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260810 01:02:59.525806 11168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:59.527156 11151 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:02:59.527343 11151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-1/wal
uuid: "f80e3559b46a4a7c89b810e26038b39e"
format_stamp: "Formatted at 2026-08-10 01:02:59 on dist-test-slave-gp6n"
server_key: "83ab807f013b3ba593060f283e72db4b"
server_key_iv: "3d5b515171d4f63565feed9ded28a173"
server_key_version: "encryptionkey@0"
I20260810 01:02:59.527555 11151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:02:59.548787 11151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:02:59.549620 11151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:02:59.549939 11151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:02:59.550904 11151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:02:59.552393 11151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:02:59.552515 11151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:59.552650 11151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:02:59.552740 11151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:59.586517 11151 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.130:41843
I20260810 01:02:59.586612 11296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.130:41843 every 8 connection(s)
I20260810 01:02:59.587958 11151 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-1/data/info.pb
I20260810 01:02:59.592484 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 11151
I20260810 01:02:59.592675 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-1/wal/instance
I20260810 01:02:59.593262 20806 external_mini_cluster.cc:1428] Setting key a981aa552b11118fb92c25021458f161
I20260810 01:02:59.596575 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.81.131:0
--local_ip_for_outbound_sockets=127.20.81.131
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:40125
--builtin_ntp_servers=127.20.81.148:39553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 01:02:59.602604 11149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:59.590545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.130:35029 (local address 127.20.81.190:40125)
0810 01:02:59.590744 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 01:02:59.590749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:02:59.592238 (+ 1489us) server_negotiation.cc:407] Connection header received
0810 01:02:59.594721 (+ 2483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:02:59.594727 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:02:59.594802 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:02:59.594909 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:02:59.596293 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:59.597146 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:59.599711 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:02:59.600080 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:02:59.600174 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:59.601789 (+ 1615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:02:59.601793 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:02:59.601796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:02:59.601816 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:02:59.601848 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:02:59.601859 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:02:59.601928 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 01:02:59.602186 (+ 258us) server_negotiation.cc:299] Negotiation successful
0810 01:02:59.602268 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 01:02:59.604053 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:02:59.591110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:40125 (local address 127.20.81.130:35029)
0810 01:02:59.592029 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0810 01:02:59.592104 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0810 01:02:59.594440 (+ 2336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:02:59.595129 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:02:59.595142 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:02:59.595266 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:02:59.596109 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:59.596134 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:59.597313 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:59.597322 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:59.599561 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:02:59.599579 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:02:59.600303 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:02:59.600312 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:02:59.600654 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:02:59.601425 (+ 771us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 01:02:59.601431 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:02:59.601485 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 01:02:59.601604 (+ 119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:02:59.601954 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:02:59.601968 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:02:59.601979 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 01:02:59.602162 (+ 183us) client_negotiation.cc:240] Negotiation successful
0810 01:02:59.602629 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":305,"threads_started":1}
I20260810 01:02:59.605600 11297 heartbeater.cc:347] Connected to a master server at 127.20.81.190:40125
I20260810 01:02:59.605924 11297 heartbeater.cc:464] Registering TS with master...
I20260810 01:02:59.606884 11297 heartbeater.cc:511] Master 127.20.81.190:40125 requested a full tablet report, sending...
I20260810 01:02:59.608430 10936 ts_manager.cc:194] Registered new tserver with Master: f80e3559b46a4a7c89b810e26038b39e (127.20.81.130:41843)
I20260810 01:02:59.609254 10936 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.81.130:35029
W20260810 01:02:59.945281 11300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:02:59.945576 11300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:02:59.945722 11300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:02:59.952934 11300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:02:59.953037 11300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:02:59.953238 11300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.81.131
I20260810 01:02:59.962466 11300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:39553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.81.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.81.131
--webserver_port=0
--tserver_master_addrs=127.20.81.190:40125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.81.131
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:02:59.963792 11300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:02:59.965592 11300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:02:59.978875 11310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:59.978896 11306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:02:59.979065 11305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:02:59.979875 11300 server_base.cc:1034] running on GCE node
I20260810 01:02:59.980440 11300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:02:59.981364 11300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:02:59.982560 11300 hybrid_clock.cc:648] HybridClock initialized: now 1786323779982482 us; error 60 us; skew 500 ppm
I20260810 01:02:59.982834 11300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:02:59.985208 11300 webserver.cc:460] Webserver started at http://127.20.81.131:34679/ using document root <none> and password file <none>
I20260810 01:02:59.985942 11300 fs_manager.cc:359] Metadata directory not provided
I20260810 01:02:59.986069 11300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:02:59.986457 11300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:02:59.989235 11300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-2/data/instance:
uuid: "3ff9de6370d842c4ab959c05c2121df6"
format_stamp: "Formatted at 2026-08-10 01:02:59 on dist-test-slave-gp6n"
server_key: "be82b7d3999ca569dafb0c5055a821db"
server_key_iv: "77d1fafdfd4a00bcefeeaf435814f924"
server_key_version: "encryptionkey@0"
I20260810 01:02:59.990016 11300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance:
uuid: "3ff9de6370d842c4ab959c05c2121df6"
format_stamp: "Formatted at 2026-08-10 01:02:59 on dist-test-slave-gp6n"
server_key: "be82b7d3999ca569dafb0c5055a821db"
server_key_iv: "77d1fafdfd4a00bcefeeaf435814f924"
server_key_version: "encryptionkey@0"
I20260810 01:02:59.994990 11300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 01:02:59.998328 11317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:02:59.999701 11300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 01:02:59.999831 11300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-2/wal
uuid: "3ff9de6370d842c4ab959c05c2121df6"
format_stamp: "Formatted at 2026-08-10 01:02:59 on dist-test-slave-gp6n"
server_key: "be82b7d3999ca569dafb0c5055a821db"
server_key_iv: "77d1fafdfd4a00bcefeeaf435814f924"
server_key_version: "encryptionkey@0"
I20260810 01:03:00.000015 11300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:00.024048 11300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:00.024839 11300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:00.025113 11300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:00.026111 11300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 01:03:00.027571 11300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:03:00.027694 11300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:00.027830 11300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 01:03:00.027928 11300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:00.062904 11300 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.131:32915
I20260810 01:03:00.062975 11446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.131:32915 every 8 connection(s)
I20260810 01:03:00.064226 11300 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-2/data/info.pb
I20260810 01:03:00.073175 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 11300
I20260810 01:03:00.073377 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-2/wal/instance
I20260810 01:03:00.074016 20806 external_mini_cluster.cc:1428] Setting key 94a89df9b3b68f43f0d1267a7f820bf1
I20260810 01:03:00.078118 11149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.066976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:51313 (local address 127.20.81.190:40125)
0810 01:03:00.067210 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:00.067220 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:00.068640 (+ 1420us) server_negotiation.cc:407] Connection header received
0810 01:03:00.070575 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:00.070580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:00.070646 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:00.070736 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:00.071875 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.072677 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.075185 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.075713 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.075860 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:00.077470 (+ 1610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:00.077475 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:00.077478 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:00.077495 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:00.077524 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:00.077536 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:00.077606 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:00.077809 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 01:03:00.077885 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 01:03:00.079327 11449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.067504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:40125 (local address 127.20.81.131:51313)
0810 01:03:00.068478 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:00.068517 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:00.070254 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:00.070955 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:00.070963 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:00.071104 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:03:00.071721 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:00.071738 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.072853 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.072861 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:00.075031 (+ 2170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:00.075048 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.075993 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.076003 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:00.076306 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:00.077099 (+ 793us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 01:03:00.077102 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:03:00.077166 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 01:03:00.077275 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:03:00.077625 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:03:00.077633 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:03:00.077639 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 01:03:00.077787 (+ 148us) client_negotiation.cc:240] Negotiation successful
0810 01:03:00.078080 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":283,"threads_started":1}
I20260810 01:03:00.080673 11447 heartbeater.cc:347] Connected to a master server at 127.20.81.190:40125
I20260810 01:03:00.081000 11447 heartbeater.cc:464] Registering TS with master...
I20260810 01:03:00.081734 11447 heartbeater.cc:511] Master 127.20.81.190:40125 requested a full tablet report, sending...
I20260810 01:03:00.083125 10936 ts_manager.cc:194] Registered new tserver with Master: 3ff9de6370d842c4ab959c05c2121df6 (127.20.81.131:32915)
I20260810 01:03:00.084023 10936 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.81.131:51313
I20260810 01:03:00.089334 20806 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 01:03:00.100441 11149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.094132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37712 (local address 127.20.81.190:40125)
0810 01:03:00.094313 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:00.094318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:00.095006 (+ 688us) server_negotiation.cc:407] Connection header received
0810 01:03:00.095140 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:00.095144 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:00.095221 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:00.095321 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:00.095809 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.096523 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.098003 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.098465 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.098561 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:00.099717 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:00.099722 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:00.099726 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:00.099745 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:00.099780 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:00.099792 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:00.099883 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:00.100051 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 01:03:00.100139 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 01:03:00.104746 10936 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 01:03:00.114095 10936 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 01:03:00.135610 11145 heartbeater.cc:503] Master 127.20.81.190:40125 was elected leader, sending a full tablet report...
I20260810 01:03:00.151705 11461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.127745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:41843 (local address 127.0.0.1:58556)
0810 01:03:00.141140 (+ 13395us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:00.141197 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:00.141489 (+ 292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:00.142875 (+ 1386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:00.142890 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:00.142960 (+ 70us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:03:00.143520 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:00.143538 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.146110 (+ 2572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.146119 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:00.148987 (+ 2868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:00.148997 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.150690 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.150696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:00.150894 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:00.150956 (+ 62us) client_negotiation.cc:769] Sending connection context
0810 01:03:00.151094 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 01:03:00.151290 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13110,"mutex_wait_us":2,"thread_start_us":12939,"threads_started":1}
I20260810 01:03:00.151676 11462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.127850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58556 (local address 127.20.81.130:41843)
0810 01:03:00.140237 (+ 12387us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:00.140251 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:00.141319 (+ 1068us) server_negotiation.cc:407] Connection header received
0810 01:03:00.141879 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:00.141888 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:00.142223 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:00.142455 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:03:00.143793 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.145869 (+ 2076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.149168 (+ 3299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.150505 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.150665 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:00.150928 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:00.151150 (+ 222us) server_negotiation.cc:299] Negotiation successful
0810 01:03:00.151320 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12140,"thread_start_us":12423,"threads_started":1}
I20260810 01:03:00.153633 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.128571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:40055 (local address 127.0.0.1:57802)
0810 01:03:00.143168 (+ 14597us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:00.143187 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:00.143352 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:00.144484 (+ 1132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:00.144492 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:00.144531 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:03:00.145091 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:00.145108 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.147815 (+ 2707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.147823 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:00.150349 (+ 2526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:00.150368 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.152765 (+ 2397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.152776 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:00.152971 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:00.153011 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 01:03:00.153123 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 01:03:00.153263 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14459,"mutex_wait_us":28,"thread_start_us":14267,"threads_started":1}
I20260810 01:03:00.153687 11465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.129164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57802 (local address 127.20.81.129:40055)
0810 01:03:00.142798 (+ 13634us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:00.142809 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:00.143302 (+ 493us) server_negotiation.cc:407] Connection header received
0810 01:03:00.143542 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:00.143550 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:00.143812 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:00.144060 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:03:00.145504 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.147602 (+ 2098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.150660 (+ 3058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.152301 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.152518 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:00.152820 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:00.153182 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 01:03:00.153370 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13438,"thread_start_us":13271,"threads_started":1}
I20260810 01:03:00.156517 11221 tablet_service.cc:1467] Processing CreateTablet for tablet 1388e895a45449b89a21f0538d7e706c (DEFAULT_TABLE table=test-table [id=2d6abfff4d274403bd4268c7dddf95d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:03:00.157825 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.143664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.131:32915 (local address 127.0.0.1:57658)
0810 01:03:00.148468 (+ 4804us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:00.148493 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:00.148635 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:00.149308 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:00.149312 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:00.149337 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:03:00.149760 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:00.149772 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.152650 (+ 2878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.152658 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:00.155367 (+ 2709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:00.155380 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.156986 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.156993 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:00.157188 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:00.157227 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 01:03:00.157362 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 01:03:00.157496 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4667,"thread_start_us":4553,"threads_started":1}
I20260810 01:03:00.157958 11466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.141932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57658 (local address 127.20.81.131:32915)
0810 01:03:00.148182 (+ 6250us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:00.148193 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:00.148625 (+ 432us) server_negotiation.cc:407] Connection header received
0810 01:03:00.148793 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:00.148801 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:00.149081 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:00.149319 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:03:00.149977 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.152534 (+ 2557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.155566 (+ 3032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.156763 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.156926 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:00.157219 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:00.157410 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 01:03:00.157592 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6062,"thread_start_us":5875,"threads_started":1}
I20260810 01:03:00.157655 11069 tablet_service.cc:1467] Processing CreateTablet for tablet 1388e895a45449b89a21f0538d7e706c (DEFAULT_TABLE table=test-table [id=2d6abfff4d274403bd4268c7dddf95d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:03:00.158529 11221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1388e895a45449b89a21f0538d7e706c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:00.161659 11373 tablet_service.cc:1467] Processing CreateTablet for tablet 1388e895a45449b89a21f0538d7e706c (DEFAULT_TABLE table=test-table [id=2d6abfff4d274403bd4268c7dddf95d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:03:00.162521 11069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1388e895a45449b89a21f0538d7e706c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:00.163033 11373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1388e895a45449b89a21f0538d7e706c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:00.174141 11469 tablet_bootstrap.cc:492] T 1388e895a45449b89a21f0538d7e706c P f80e3559b46a4a7c89b810e26038b39e: Bootstrap starting.
I20260810 01:03:00.176357 11470 tablet_bootstrap.cc:492] T 1388e895a45449b89a21f0538d7e706c P c75d08fa4f5d433691dd2aa7d9aae9c6: Bootstrap starting.
I20260810 01:03:00.178009 11471 tablet_bootstrap.cc:492] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6: Bootstrap starting.
I20260810 01:03:00.179499 11469 tablet_bootstrap.cc:654] T 1388e895a45449b89a21f0538d7e706c P f80e3559b46a4a7c89b810e26038b39e: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:00.180294 11470 tablet_bootstrap.cc:654] T 1388e895a45449b89a21f0538d7e706c P c75d08fa4f5d433691dd2aa7d9aae9c6: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:00.181006 11469 log.cc:824] T 1388e895a45449b89a21f0538d7e706c P f80e3559b46a4a7c89b810e26038b39e: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:00.181392 11470 log.cc:824] T 1388e895a45449b89a21f0538d7e706c P c75d08fa4f5d433691dd2aa7d9aae9c6: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:00.182631 11471 tablet_bootstrap.cc:654] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:00.183905 11471 log.cc:824] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:00.184634 11469 tablet_bootstrap.cc:492] T 1388e895a45449b89a21f0538d7e706c P f80e3559b46a4a7c89b810e26038b39e: No bootstrap required, opened a new log
I20260810 01:03:00.184904 11470 tablet_bootstrap.cc:492] T 1388e895a45449b89a21f0538d7e706c P c75d08fa4f5d433691dd2aa7d9aae9c6: No bootstrap required, opened a new log
I20260810 01:03:00.185045 11469 ts_tablet_manager.cc:1397] T 1388e895a45449b89a21f0538d7e706c P f80e3559b46a4a7c89b810e26038b39e: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260810 01:03:00.185215 11470 ts_tablet_manager.cc:1397] T 1388e895a45449b89a21f0538d7e706c P c75d08fa4f5d433691dd2aa7d9aae9c6: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260810 01:03:00.187366 11471 tablet_bootstrap.cc:492] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6: No bootstrap required, opened a new log
I20260810 01:03:00.187636 11471 ts_tablet_manager.cc:1397] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.006s
I20260810 01:03:00.194116 11469 raft_consensus.cc:348] T 1388e895a45449b89a21f0538d7e706c P f80e3559b46a4a7c89b810e26038b39e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e3559b46a4a7c89b810e26038b39e" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41843 } } peers { permanent_uuid: "3ff9de6370d842c4ab959c05c2121df6" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 32915 } } peers { permanent_uuid: "c75d08fa4f5d433691dd2aa7d9aae9c6" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40055 } }
I20260810 01:03:00.194530 11469 raft_consensus.cc:374] T 1388e895a45449b89a21f0538d7e706c P f80e3559b46a4a7c89b810e26038b39e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:03:00.194653 11469 raft_consensus.cc:729] T 1388e895a45449b89a21f0538d7e706c P f80e3559b46a4a7c89b810e26038b39e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f80e3559b46a4a7c89b810e26038b39e, State: Initialized, Role: FOLLOWER
I20260810 01:03:00.195325 11469 consensus_queue.cc:260] T 1388e895a45449b89a21f0538d7e706c P f80e3559b46a4a7c89b810e26038b39e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e3559b46a4a7c89b810e26038b39e" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41843 } } peers { permanent_uuid: "3ff9de6370d842c4ab959c05c2121df6" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 32915 } } peers { permanent_uuid: "c75d08fa4f5d433691dd2aa7d9aae9c6" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40055 } }
I20260810 01:03:00.196557 11297 heartbeater.cc:503] Master 127.20.81.190:40125 was elected leader, sending a full tablet report...
I20260810 01:03:00.197153 11469 ts_tablet_manager.cc:1428] T 1388e895a45449b89a21f0538d7e706c P f80e3559b46a4a7c89b810e26038b39e: Time spent starting tablet: real 0.012s user 0.005s sys 0.005s
I20260810 01:03:00.198047 11470 raft_consensus.cc:348] T 1388e895a45449b89a21f0538d7e706c P c75d08fa4f5d433691dd2aa7d9aae9c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e3559b46a4a7c89b810e26038b39e" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41843 } } peers { permanent_uuid: "3ff9de6370d842c4ab959c05c2121df6" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 32915 } } peers { permanent_uuid: "c75d08fa4f5d433691dd2aa7d9aae9c6" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40055 } }
I20260810 01:03:00.198616 11470 raft_consensus.cc:374] T 1388e895a45449b89a21f0538d7e706c P c75d08fa4f5d433691dd2aa7d9aae9c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:03:00.198740 11470 raft_consensus.cc:729] T 1388e895a45449b89a21f0538d7e706c P c75d08fa4f5d433691dd2aa7d9aae9c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75d08fa4f5d433691dd2aa7d9aae9c6, State: Initialized, Role: FOLLOWER
I20260810 01:03:00.199060 11471 raft_consensus.cc:348] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e3559b46a4a7c89b810e26038b39e" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41843 } } peers { permanent_uuid: "3ff9de6370d842c4ab959c05c2121df6" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 32915 } } peers { permanent_uuid: "c75d08fa4f5d433691dd2aa7d9aae9c6" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40055 } }
I20260810 01:03:00.199530 11471 raft_consensus.cc:374] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:03:00.199420 11470 consensus_queue.cc:260] T 1388e895a45449b89a21f0538d7e706c P c75d08fa4f5d433691dd2aa7d9aae9c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e3559b46a4a7c89b810e26038b39e" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41843 } } peers { permanent_uuid: "3ff9de6370d842c4ab959c05c2121df6" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 32915 } } peers { permanent_uuid: "c75d08fa4f5d433691dd2aa7d9aae9c6" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40055 } }
I20260810 01:03:00.199658 11471 raft_consensus.cc:729] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ff9de6370d842c4ab959c05c2121df6, State: Initialized, Role: FOLLOWER
I20260810 01:03:00.200462 11471 consensus_queue.cc:260] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e3559b46a4a7c89b810e26038b39e" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41843 } } peers { permanent_uuid: "3ff9de6370d842c4ab959c05c2121df6" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 32915 } } peers { permanent_uuid: "c75d08fa4f5d433691dd2aa7d9aae9c6" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40055 } }
I20260810 01:03:00.201416 11470 ts_tablet_manager.cc:1428] T 1388e895a45449b89a21f0538d7e706c P c75d08fa4f5d433691dd2aa7d9aae9c6: Time spent starting tablet: real 0.016s user 0.013s sys 0.001s
I20260810 01:03:00.202554 11447 heartbeater.cc:503] Master 127.20.81.190:40125 was elected leader, sending a full tablet report...
I20260810 01:03:00.203816 11471 ts_tablet_manager.cc:1428] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260810 01:03:00.297130 11479 raft_consensus.cc:482] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:03:00.297600 11479 raft_consensus.cc:504] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e3559b46a4a7c89b810e26038b39e" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41843 } } peers { permanent_uuid: "3ff9de6370d842c4ab959c05c2121df6" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 32915 } } peers { permanent_uuid: "c75d08fa4f5d433691dd2aa7d9aae9c6" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40055 } }
I20260810 01:03:00.299621 11479 leader_election.cc:288] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f80e3559b46a4a7c89b810e26038b39e (127.20.81.130:41843), c75d08fa4f5d433691dd2aa7d9aae9c6 (127.20.81.129:40055)
I20260810 01:03:00.306615 11449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.299957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.130:41843 (local address 127.20.81.131:37655)
0810 01:03:00.300313 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:00.300335 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:00.300533 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:00.300992 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:00.300997 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:00.301024 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:03:00.301271 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:00.301283 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.302619 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.302625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:00.304396 (+ 1771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:00.304408 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.305747 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.305753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:00.305966 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:00.306002 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 01:03:00.306103 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 01:03:00.306252 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":1}
I20260810 01:03:00.306609 11462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.300106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:37655 (local address 127.20.81.130:41843)
0810 01:03:00.300361 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:00.300371 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:00.300431 (+ 60us) server_negotiation.cc:407] Connection header received
0810 01:03:00.300715 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:00.300722 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:00.300844 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:00.300967 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:03:00.301452 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.302481 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.304569 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.305585 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.305711 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:00.305762 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:00.306154 (+ 392us) server_negotiation.cc:299] Negotiation successful
0810 01:03:00.306252 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 01:03:00.307340 11465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.300106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.81.131:42149 (local address 127.20.81.129:40055)
0810 01:03:00.300315 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:00.300322 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:00.300946 (+ 624us) server_negotiation.cc:407] Connection header received
0810 01:03:00.301285 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:00.301290 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:00.301392 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:00.301489 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 01:03:00.302090 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.303056 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.305230 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.306266 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.306461 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:00.306525 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:00.306938 (+ 413us) server_negotiation.cc:299] Negotiation successful
0810 01:03:00.307085 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 01:03:00.307893 11480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.300045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:40055 (local address 127.20.81.131:42149)
0810 01:03:00.300832 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:00.300850 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:00.301058 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:00.301532 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:00.301538 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:00.301566 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 01:03:00.301938 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:00.301950 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.303214 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.303218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:00.305033 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:00.305047 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.306465 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.306474 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:00.306675 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:00.306708 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 01:03:00.307531 (+ 823us) client_negotiation.cc:240] Negotiation successful
0810 01:03:00.307638 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":4,"thread_start_us":450,"threads_started":1}
I20260810 01:03:00.307813 11244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1388e895a45449b89a21f0538d7e706c" candidate_uuid: "3ff9de6370d842c4ab959c05c2121df6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f80e3559b46a4a7c89b810e26038b39e" is_pre_election: true
I20260810 01:03:00.308446 11244 raft_consensus.cc:2454] T 1388e895a45449b89a21f0538d7e706c P f80e3559b46a4a7c89b810e26038b39e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ff9de6370d842c4ab959c05c2121df6 in term 0.
I20260810 01:03:00.308983 11089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1388e895a45449b89a21f0538d7e706c" candidate_uuid: "3ff9de6370d842c4ab959c05c2121df6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75d08fa4f5d433691dd2aa7d9aae9c6" is_pre_election: true
I20260810 01:03:00.309201 11319 leader_election.cc:302] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ff9de6370d842c4ab959c05c2121df6, f80e3559b46a4a7c89b810e26038b39e; no voters:
I20260810 01:03:00.309594 11089 raft_consensus.cc:2454] T 1388e895a45449b89a21f0538d7e706c P c75d08fa4f5d433691dd2aa7d9aae9c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ff9de6370d842c4ab959c05c2121df6 in term 0.
I20260810 01:03:00.309649 11479 raft_consensus.cc:2781] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:03:00.309808 11479 raft_consensus.cc:482] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:03:00.309926 11479 raft_consensus.cc:3037] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:00.312668 11479 raft_consensus.cc:504] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e3559b46a4a7c89b810e26038b39e" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41843 } } peers { permanent_uuid: "3ff9de6370d842c4ab959c05c2121df6" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 32915 } } peers { permanent_uuid: "c75d08fa4f5d433691dd2aa7d9aae9c6" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40055 } }
I20260810 01:03:00.313418 11479 leader_election.cc:288] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6 [CANDIDATE]: Term 1 election: Requested vote from peers f80e3559b46a4a7c89b810e26038b39e (127.20.81.130:41843), c75d08fa4f5d433691dd2aa7d9aae9c6 (127.20.81.129:40055)
I20260810 01:03:00.313903 11244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1388e895a45449b89a21f0538d7e706c" candidate_uuid: "3ff9de6370d842c4ab959c05c2121df6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f80e3559b46a4a7c89b810e26038b39e"
I20260810 01:03:00.313972 11089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1388e895a45449b89a21f0538d7e706c" candidate_uuid: "3ff9de6370d842c4ab959c05c2121df6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75d08fa4f5d433691dd2aa7d9aae9c6"
I20260810 01:03:00.314155 11089 raft_consensus.cc:3037] T 1388e895a45449b89a21f0538d7e706c P c75d08fa4f5d433691dd2aa7d9aae9c6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:00.314146 11244 raft_consensus.cc:3037] T 1388e895a45449b89a21f0538d7e706c P f80e3559b46a4a7c89b810e26038b39e [term 0 FOLLOWER]: Advancing to term 1
W20260810 01:03:00.315902 11448 tablet.cc:2369] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:03:00.317551 11089 raft_consensus.cc:2454] T 1388e895a45449b89a21f0538d7e706c P c75d08fa4f5d433691dd2aa7d9aae9c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ff9de6370d842c4ab959c05c2121df6 in term 1.
I20260810 01:03:00.317565 11244 raft_consensus.cc:2454] T 1388e895a45449b89a21f0538d7e706c P f80e3559b46a4a7c89b810e26038b39e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ff9de6370d842c4ab959c05c2121df6 in term 1.
I20260810 01:03:00.318002 11319 leader_election.cc:302] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ff9de6370d842c4ab959c05c2121df6, f80e3559b46a4a7c89b810e26038b39e; no voters:
I20260810 01:03:00.318346 11479 raft_consensus.cc:2781] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:03:00.319079 11479 raft_consensus.cc:686] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6 [term 1 LEADER]: Becoming Leader. State: Replica: 3ff9de6370d842c4ab959c05c2121df6, State: Running, Role: LEADER
I20260810 01:03:00.319654 11479 consensus_queue.cc:237] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e3559b46a4a7c89b810e26038b39e" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41843 } } peers { permanent_uuid: "3ff9de6370d842c4ab959c05c2121df6" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 32915 } } peers { permanent_uuid: "c75d08fa4f5d433691dd2aa7d9aae9c6" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40055 } }
I20260810 01:03:00.324364 10936 catalog_manager.cc:5495] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ff9de6370d842c4ab959c05c2121df6 (127.20.81.131). New cstate: current_term: 1 leader_uuid: "3ff9de6370d842c4ab959c05c2121df6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e3559b46a4a7c89b810e26038b39e" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ff9de6370d842c4ab959c05c2121df6" member_type: VOTER last_known_addr { host: "127.20.81.131" port: 32915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c75d08fa4f5d433691dd2aa7d9aae9c6" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40055 } health_report { overall_health: UNKNOWN } } }
W20260810 01:03:00.340152 11298 tablet.cc:2369] T 1388e895a45449b89a21f0538d7e706c P f80e3559b46a4a7c89b810e26038b39e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 01:03:00.360669 11146 tablet.cc:2369] T 1388e895a45449b89a21f0538d7e706c P c75d08fa4f5d433691dd2aa7d9aae9c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:03:00.377792 11466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.372533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57662 (local address 127.20.81.131:32915)
0810 01:03:00.372712 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:00.372716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:00.372734 (+ 18us) server_negotiation.cc:407] Connection header received
0810 01:03:00.372845 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:00.372849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:00.372934 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:00.373069 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:00.373479 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.374225 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.375650 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.376060 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.376188 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:00.376701 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:00.377228 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:00.377401 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:00.377462 (+ 61us) server_negotiation.cc:299] Negotiation successful
0810 01:03:00.377541 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 01:03:00.386826 11244 raft_consensus.cc:1261] T 1388e895a45449b89a21f0538d7e706c P f80e3559b46a4a7c89b810e26038b39e [term 1 FOLLOWER]: Refusing update from remote peer 3ff9de6370d842c4ab959c05c2121df6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:03:00.386826 11089 raft_consensus.cc:1261] T 1388e895a45449b89a21f0538d7e706c P c75d08fa4f5d433691dd2aa7d9aae9c6 [term 1 FOLLOWER]: Refusing update from remote peer 3ff9de6370d842c4ab959c05c2121df6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:03:00.388058 11481 consensus_queue.cc:1035] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f80e3559b46a4a7c89b810e26038b39e" member_type: VOTER last_known_addr { host: "127.20.81.130" port: 41843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:03:00.388454 11479 consensus_queue.cc:1035] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75d08fa4f5d433691dd2aa7d9aae9c6" member_type: VOTER last_known_addr { host: "127.20.81.129" port: 40055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:03:00.401227 11486 mvcc.cc:205] Tried to move back new op lower bound from 7316782204452274176 to 7316782204193439744. Current Snapshot: MvccSnapshot[applied={T|T < 7316782204452274176}]
I20260810 01:03:00.418282 10935 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:03:00.418645 10935 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 01:03:00.422633 10935 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9374ce40127b48d5951b53a3fa21569d: Sending DeleteTablet for 3 replicas of tablet 1388e895a45449b89a21f0538d7e706c
I20260810 01:03:00.424015 11069 tablet_service.cc:1514] Processing DeleteTablet for tablet 1388e895a45449b89a21f0538d7e706c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:03:00 UTC) from {username='slave'} at 127.0.0.1:57802
I20260810 01:03:00.424010 11221 tablet_service.cc:1514] Processing DeleteTablet for tablet 1388e895a45449b89a21f0538d7e706c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:03:00 UTC) from {username='slave'} at 127.0.0.1:58556
I20260810 01:03:00.424515 11373 tablet_service.cc:1514] Processing DeleteTablet for tablet 1388e895a45449b89a21f0538d7e706c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 01:03:00 UTC) from {username='slave'} at 127.0.0.1:57658
I20260810 01:03:00.429909 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 11001
I20260810 01:03:00.435096 11498 tablet_replica.cc:330] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6: stopping tablet replica
I20260810 01:03:00.436312 11498 raft_consensus.cc:2229] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6 [term 1 LEADER]: Raft consensus shutting down.
I20260810 01:03:00.437753 11498 raft_consensus.cc:2258] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:03:00.441275 11498 ts_tablet_manager.cc:1905] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:03:00.442070 11497 tablet_replica.cc:330] T 1388e895a45449b89a21f0538d7e706c P c75d08fa4f5d433691dd2aa7d9aae9c6: stopping tablet replica
I20260810 01:03:00.443140 11496 tablet_replica.cc:330] T 1388e895a45449b89a21f0538d7e706c P f80e3559b46a4a7c89b810e26038b39e: stopping tablet replica
I20260810 01:03:00.443636 11496 raft_consensus.cc:2229] T 1388e895a45449b89a21f0538d7e706c P f80e3559b46a4a7c89b810e26038b39e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:03:00.444003 11497 raft_consensus.cc:2229] T 1388e895a45449b89a21f0538d7e706c P c75d08fa4f5d433691dd2aa7d9aae9c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:03:00.444600 11496 raft_consensus.cc:2258] T 1388e895a45449b89a21f0538d7e706c P f80e3559b46a4a7c89b810e26038b39e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:03:00.445122 11465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.431537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57816 (local address 127.20.81.129:40055)
0810 01:03:00.431732 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:00.431738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:00.434345 (+ 2607us) server_negotiation.cc:407] Connection header received
0810 01:03:00.435137 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:00.435144 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:00.435281 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:00.435378 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:00.436181 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.437467 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.439658 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.440343 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.440449 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:00.443329 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:00.443341 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:00.443357 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:00.444172 (+ 815us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:00.444244 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:00.444269 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:00.444352 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:00.444607 (+ 255us) server_negotiation.cc:299] Negotiation successful
0810 01:03:00.444731 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 01:03:00.445811 11497 raft_consensus.cc:2258] T 1388e895a45449b89a21f0538d7e706c P c75d08fa4f5d433691dd2aa7d9aae9c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:03:00.446235 11137 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:03:00.448509 11496 ts_tablet_manager.cc:1905] T 1388e895a45449b89a21f0538d7e706c P f80e3559b46a4a7c89b810e26038b39e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:03:00.449774 11498 ts_tablet_manager.cc:1918] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:03:00.450047 11498 log.cc:1196] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-2/wal/wals/1388e895a45449b89a21f0538d7e706c
I20260810 01:03:00.450778 11498 ts_tablet_manager.cc:1939] T 1388e895a45449b89a21f0538d7e706c P 3ff9de6370d842c4ab959c05c2121df6: Deleting consensus metadata
I20260810 01:03:00.452791 10916 catalog_manager.cc:4841] TS 3ff9de6370d842c4ab959c05c2121df6 (127.20.81.131:32915): tablet 1388e895a45449b89a21f0538d7e706c (table test-table [id=2d6abfff4d274403bd4268c7dddf95d8]) successfully deleted
I20260810 01:03:00.457949 11496 ts_tablet_manager.cc:1918] T 1388e895a45449b89a21f0538d7e706c P f80e3559b46a4a7c89b810e26038b39e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 01:03:00.458114 11496 log.cc:1196] T 1388e895a45449b89a21f0538d7e706c P f80e3559b46a4a7c89b810e26038b39e: Deleting WAL directory at /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323445691443-20806-0/minicluster-data/ts-1/wal/wals/1388e895a45449b89a21f0538d7e706c
I20260810 01:03:00.458696 11496 ts_tablet_manager.cc:1939] T 1388e895a45449b89a21f0538d7e706c P f80e3559b46a4a7c89b810e26038b39e: Deleting consensus metadata
I20260810 01:03:00.460536 10916 catalog_manager.cc:4841] TS f80e3559b46a4a7c89b810e26038b39e (127.20.81.130:41843): tablet 1388e895a45449b89a21f0538d7e706c (table test-table [id=2d6abfff4d274403bd4268c7dddf95d8]) successfully deleted
I20260810 01:03:00.545140 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 11001
W20260810 01:03:00.569160 10917 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.81.129:40055 (error 108)
I20260810 01:03:00.569993 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 11151
I20260810 01:03:00.571416 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.570833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:40055 (local address 127.0.0.1:57822)
0810 01:03:00.571049 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:00.571192 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:40055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260810 01:03:00.571682 10917 catalog_manager.cc:4584] TS c75d08fa4f5d433691dd2aa7d9aae9c6 (127.20.81.129:40055): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1388e895a45449b89a21f0538d7e706c: Network error: Client connection negotiation failed: client connection to 127.20.81.129:40055: connect: Connection refused (error 111)
I20260810 01:03:00.578603 11462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.570855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58570 (local address 127.20.81.130:41843)
0810 01:03:00.571155 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:00.571161 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:00.571186 (+ 25us) server_negotiation.cc:407] Connection header received
0810 01:03:00.571400 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:00.571407 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:00.571533 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:00.571641 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:00.572172 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.573336 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.574981 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.575641 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.575774 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:00.576909 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:00.576923 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:00.576941 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:00.577719 (+ 778us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:00.577789 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:00.577814 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:00.577928 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:00.578061 (+ 133us) server_negotiation.cc:299] Negotiation successful
0810 01:03:00.578176 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166}
I20260810 01:03:00.579293 11292 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:03:00.626058 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.625352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:40055 (local address 127.0.0.1:57830)
0810 01:03:00.625617 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:00.625751 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:40055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260810 01:03:00.627876 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.627431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:40055 (local address 127.0.0.1:57844)
0810 01:03:00.627649 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:00.627738 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:40055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260810 01:03:00.671936 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 11151
I20260810 01:03:00.688616 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.688008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:40055 (local address 127.0.0.1:57858)
0810 01:03:00.688259 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:00.688388 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:40055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260810 01:03:00.689993 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.689505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:40055 (local address 127.0.0.1:57870)
0810 01:03:00.689707 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:00.689810 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:40055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 01:03:00.694537 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 11300
I20260810 01:03:00.703102 11466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.695562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57672 (local address 127.20.81.131:32915)
0810 01:03:00.695773 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:00.695778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:00.695800 (+ 22us) server_negotiation.cc:407] Connection header received
0810 01:03:00.695874 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:00.695880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:00.696010 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:00.696133 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:00.696625 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.697862 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.699399 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:00.700105 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:00.700242 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:00.701385 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:00.701397 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:00.701416 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:00.702267 (+ 851us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:00.702326 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:00.702351 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:00.702531 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:00.702613 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 01:03:00.702713 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 01:03:00.703986 11440 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:03:00.770689 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.770082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:40055 (local address 127.0.0.1:57876)
0810 01:03:00.770329 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:00.770506 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:40055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260810 01:03:00.771791 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.771394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:40055 (local address 127.0.0.1:57886)
0810 01:03:00.771596 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:00.771684 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:40055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260810 01:03:00.826002 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 11300
I20260810 01:03:00.850689 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 10900
I20260810 01:03:00.851871 10967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:03:00.965981 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:00.965086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.129:40055 (local address 127.0.0.1:57896)
0810 01:03:00.965332 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:00.965420 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.129:40055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 01:03:00.966012 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 10900
2026-08-10T01:03:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2996 ms)
[----------] 6 tests from EncryptionPolicyTest (24103 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:03:01 dist-test-slave-gp6n krb5kdc[11508](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:03:01 dist-test-slave-gp6n krb5kdc[11508](info): set up 2 sockets
Aug 10 01:03:01 dist-test-slave-gp6n krb5kdc[11508](info): commencing operation
krb5kdc: starting...
W20260810 01:03:02.219056 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:03:02 dist-test-slave-gp6n krb5kdc[11508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:03:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:03:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 01:03:02.562319 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35973
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:03:02.911031 11527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:02.911329 11527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:02.911494 11527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:02.918793 11527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:02.918933 11527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:02.919023 11527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:02.919126 11527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:02.919206 11527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:02.929004 11527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35973
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:02.930434 11527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:02.931772 11527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:02.943928 11534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:02.944003 11539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:02.944036 11535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:02.944700 11527 server_base.cc:1034] running on GCE node
I20260810 01:03:02.945564 11527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:02.947086 11527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:02.948295 11527 hybrid_clock.cc:648] HybridClock initialized: now 1786323782948253 us; error 30 us; skew 500 ppm
I20260810 01:03:02.948678 11527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:02 dist-test-slave-gp6n krb5kdc[11508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323782, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:02.956117 11527 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 01:03:02.957624 11527 webserver.cc:460] Webserver started at http://127.0.0.1:46049/ using document root <none> and password file <none>
I20260810 01:03:02.958305 11527 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:02.958485 11527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:02.958768 11527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:02.961798 11527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "18f542d503b94b2ba675c9ecdb434d24"
format_stamp: "Formatted at 2026-08-10 01:03:02 on dist-test-slave-gp6n"
server_key: "3c8a8240e277bf3b83b675bf2c21307b"
server_key_iv: "a9a083dbaeb3fc8ce7ccf941b9f8da1d"
server_key_version: "encryptionkey@0"
I20260810 01:03:02.962627 11527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "18f542d503b94b2ba675c9ecdb434d24"
format_stamp: "Formatted at 2026-08-10 01:03:02 on dist-test-slave-gp6n"
server_key: "3c8a8240e277bf3b83b675bf2c21307b"
server_key_iv: "a9a083dbaeb3fc8ce7ccf941b9f8da1d"
server_key_version: "encryptionkey@0"
I20260810 01:03:02.967883 11527 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 01:03:02.971379 11548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:02.972751 11527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 01:03:02.972954 11527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "18f542d503b94b2ba675c9ecdb434d24"
format_stamp: "Formatted at 2026-08-10 01:03:02 on dist-test-slave-gp6n"
server_key: "3c8a8240e277bf3b83b675bf2c21307b"
server_key_iv: "a9a083dbaeb3fc8ce7ccf941b9f8da1d"
server_key_version: "encryptionkey@0"
I20260810 01:03:02.973168 11527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:02.992834 11527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:02.997153 11527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:02.997394 11527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:03.016199 11527 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35973
I20260810 01:03:03.016233 11607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35973 every 8 connection(s)
I20260810 01:03:03.017582 11527 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:03:03.021380 11609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:03.024362 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 11527
I20260810 01:03:03.024576 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:03:03.025274 20806 external_mini_cluster.cc:1428] Setting key 16a0a86ac85d9511a99c5f95060b1a51
I20260810 01:03:03.034654 11609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24: Bootstrap starting.
I20260810 01:03:03.038316 11609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:03.039355 11609 log.cc:824] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24: Log is configured to *not* fsync() on all Append() calls
Aug 10 01:03:03 dist-test-slave-gp6n krb5kdc[11508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 01:03:03.042675 11609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24: No bootstrap required, opened a new log
I20260810 01:03:03.050169 11612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:03.027318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54855 (local address 127.0.0.1:35973)
0810 01:03:03.028182 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:03.028198 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:03.028253 (+ 55us) server_negotiation.cc:407] Connection header received
0810 01:03:03.030461 (+ 2208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:03.030492 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:03.031049 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:03.031498 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:03.032885 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:03.034919 (+ 2034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:03.036191 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:03.037225 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:03.037403 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:03.042166 (+ 4763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:03.042173 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:03.042187 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:03.042226 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:03.045580 (+ 3354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:03.046594 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:03.046600 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:03.046606 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:03.046787 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:03.047314 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:03.047318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:03.047320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:03.047873 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:03.048133 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:03.048659 (+ 526us) server_negotiation.cc:299] Negotiation successful
0810 01:03:03.049009 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":339,"threads_started":1}
I20260810 01:03:03.052016 11609 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f542d503b94b2ba675c9ecdb434d24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35973 } }
I20260810 01:03:03.052402 11609 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:03:03.052533 11609 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18f542d503b94b2ba675c9ecdb434d24, State: Initialized, Role: FOLLOWER
I20260810 01:03:03.053196 11609 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f542d503b94b2ba675c9ecdb434d24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35973 } }
I20260810 01:03:03.053495 11609 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:03:03.053656 11609 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:03:03.053889 11609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:03.054926 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 01:03:03.057291 11609 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f542d503b94b2ba675c9ecdb434d24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35973 } }
I20260810 01:03:03.057948 11609 leader_election.cc:302] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18f542d503b94b2ba675c9ecdb434d24; no voters:
I20260810 01:03:03.058710 11609 leader_election.cc:288] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:03:03.058864 11614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:03:03.060289 11614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24 [term 1 LEADER]: Becoming Leader. State: Replica: 18f542d503b94b2ba675c9ecdb434d24, State: Running, Role: LEADER
I20260810 01:03:03.060997 11614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f542d503b94b2ba675c9ecdb434d24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35973 } }
I20260810 01:03:03.061463 11609 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:03.063920 11619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18f542d503b94b2ba675c9ecdb434d24. Latest consensus state: current_term: 1 leader_uuid: "18f542d503b94b2ba675c9ecdb434d24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f542d503b94b2ba675c9ecdb434d24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35973 } } }
I20260810 01:03:03.064394 11619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:03.065083 11617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18f542d503b94b2ba675c9ecdb434d24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f542d503b94b2ba675c9ecdb434d24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35973 } } }
I20260810 01:03:03.065436 11617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:03.067085 11624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:03:03.073868 11624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:03:03.085623 11624 catalog_manager.cc:1347] Generated new cluster ID: 4fab882686e54cf0991f190de9a910fd
I20260810 01:03:03.085731 11624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:03:03.089350 11612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:03.063348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54651 (local address 127.0.0.1:35973)
0810 01:03:03.063579 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:03.063588 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:03.067780 (+ 4192us) server_negotiation.cc:407] Connection header received
0810 01:03:03.068000 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:03.068011 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:03.068122 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:03.068242 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:03.070009 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:03.071267 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:03.073183 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:03.073886 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:03.073990 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:03.078020 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:03.078029 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:03.078036 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:03.078094 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:03.084312 (+ 6218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:03.085675 (+ 1363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:03.085684 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:03.085690 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:03.085902 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:03.086660 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:03.086670 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:03.086676 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:03.087238 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:03.087547 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:03.088499 (+ 952us) server_negotiation.cc:299] Negotiation successful
0810 01:03:03.088757 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 01:03:03.106232 11624 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:03:03.107550 11624 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:03:03.113865 11624 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18f542d503b94b2ba675c9ecdb434d24: Generated new TSK 0
I20260810 01:03:03.114719 11624 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:03:03.126219 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 11527
I20260810 01:03:03.127606 11602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:03:03.219352 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 11527
W20260810 01:03:03.241266 11506 connection.cc:535] client connection to 127.0.0.1:35973 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35973): Connection reset by peer (error 104)
2026-08-10T01:03:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2245 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:03:03 dist-test-slave-gp6n krb5kdc[11643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:03:03 dist-test-slave-gp6n krb5kdc[11643](info): set up 2 sockets
Aug 10 01:03:03 dist-test-slave-gp6n krb5kdc[11643](info): commencing operation
krb5kdc: starting...
W20260810 01:03:04.431531 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.005s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:03:04 dist-test-slave-gp6n krb5kdc[11643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:03:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:03:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 01:03:04.781435 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44127
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:03:05.135774 11657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:05.136112 11657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:05.136214 11657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:05.144348 11657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:05.144479 11657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:05.144532 11657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:05.144603 11657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:05.144651 11657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:05.155494 11657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44127
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:05.156844 11657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:05.158147 11657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:05.170496 11667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:05.170611 11664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:05.170580 11665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:05.171618 11657 server_base.cc:1034] running on GCE node
I20260810 01:03:05.172438 11657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:05.174002 11657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:05.175407 11657 hybrid_clock.cc:648] HybridClock initialized: now 1786323785175339 us; error 50 us; skew 500 ppm
I20260810 01:03:05.175832 11657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:05 dist-test-slave-gp6n krb5kdc[11643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323785, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:05.179293 11657 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 01:03:05.180804 11657 webserver.cc:460] Webserver started at http://127.0.0.1:40823/ using document root <none> and password file <none>
I20260810 01:03:05.181505 11657 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:05.181659 11657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:05.181979 11657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:05.185081 11657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "be04af34f168433e8d067a0a12725de8"
format_stamp: "Formatted at 2026-08-10 01:03:05 on dist-test-slave-gp6n"
server_key: "a011980e8aab22c69fbfe2b854c68f72"
server_key_iv: "0127596693ca2a28291e09cb1ad5de25"
server_key_version: "encryptionkey@0"
I20260810 01:03:05.185884 11657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "be04af34f168433e8d067a0a12725de8"
format_stamp: "Formatted at 2026-08-10 01:03:05 on dist-test-slave-gp6n"
server_key: "a011980e8aab22c69fbfe2b854c68f72"
server_key_iv: "0127596693ca2a28291e09cb1ad5de25"
server_key_version: "encryptionkey@0"
I20260810 01:03:05.191201 11657 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 01:03:05.194796 11675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:05.196094 11657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 01:03:05.196281 11657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "be04af34f168433e8d067a0a12725de8"
format_stamp: "Formatted at 2026-08-10 01:03:05 on dist-test-slave-gp6n"
server_key: "a011980e8aab22c69fbfe2b854c68f72"
server_key_iv: "0127596693ca2a28291e09cb1ad5de25"
server_key_version: "encryptionkey@0"
I20260810 01:03:05.196487 11657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:05.211477 11657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:05.215905 11657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:05.216281 11657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:05.241183 11657 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44127
I20260810 01:03:05.241185 11731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44127 every 8 connection(s)
I20260810 01:03:05.242947 11657 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:03:05.244009 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 11657
I20260810 01:03:05.244236 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:03:05.244935 20806 external_mini_cluster.cc:1428] Setting key 8a3bb224a08108ecb595c8927eeca558
I20260810 01:03:05.247625 11732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:05.261996 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:05.247097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57213 (local address 127.0.0.1:44127)
0810 01:03:05.247904 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:05.247923 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:05.247988 (+ 65us) server_negotiation.cc:407] Connection header received
0810 01:03:05.250345 (+ 2357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:05.250372 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:05.250945 (+ 573us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:05.251418 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:05.252082 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:05.254144 (+ 2062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:05.256479 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:05.257591 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:05.257790 (+ 199us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:05.259058 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:05.259069 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:05.259087 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:05.259875 (+ 788us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:05.259980 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:05.260016 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:05.260169 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:05.260481 (+ 312us) server_negotiation.cc:299] Negotiation successful
0810 01:03:05.260645 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":329,"threads_started":1}
I20260810 01:03:05.262230 11732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8: Bootstrap starting.
I20260810 01:03:05.265978 11732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:05.266556 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 01:03:05.267184 11732 log.cc:824] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:05.269942 11732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8: No bootstrap required, opened a new log
I20260810 01:03:05.278714 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:05.271593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40647 (local address 127.0.0.1:44127)
0810 01:03:05.271778 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:05.271784 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:05.272389 (+ 605us) server_negotiation.cc:407] Connection header received
0810 01:03:05.272542 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:05.272546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:05.272619 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:05.272709 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:05.273268 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:05.274094 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:05.276051 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:05.276508 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:05.276605 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:05.277862 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:05.277867 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:05.277870 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:05.277886 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:05.277926 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:05.277940 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:05.278021 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:05.278246 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 01:03:05.278289 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 01:03:05.280149 11732 raft_consensus.cc:348] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be04af34f168433e8d067a0a12725de8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44127 } }
I20260810 01:03:05.280633 11732 raft_consensus.cc:374] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:03:05.280745 11732 raft_consensus.cc:729] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be04af34f168433e8d067a0a12725de8, State: Initialized, Role: FOLLOWER
I20260810 01:03:05.281302 11732 consensus_queue.cc:260] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be04af34f168433e8d067a0a12725de8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44127 } }
I20260810 01:03:05.281548 11732 raft_consensus.cc:388] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:03:05.281702 11732 raft_consensus.cc:482] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:03:05.281889 11732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:05.284543 11732 raft_consensus.cc:504] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be04af34f168433e8d067a0a12725de8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44127 } }
I20260810 01:03:05.285077 11732 leader_election.cc:302] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be04af34f168433e8d067a0a12725de8; no voters:
I20260810 01:03:05.285668 11732 leader_election.cc:288] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:03:05.285821 11744 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:03:05.286633 11744 raft_consensus.cc:686] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8 [term 1 LEADER]: Becoming Leader. State: Replica: be04af34f168433e8d067a0a12725de8, State: Running, Role: LEADER
I20260810 01:03:05.287209 11744 consensus_queue.cc:237] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be04af34f168433e8d067a0a12725de8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44127 } }
I20260810 01:03:05.287948 11732 sys_catalog.cc:564] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:05.290365 11745 sys_catalog.cc:455] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be04af34f168433e8d067a0a12725de8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be04af34f168433e8d067a0a12725de8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44127 } } }
I20260810 01:03:05.290870 11745 sys_catalog.cc:458] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:05.292634 11747 sys_catalog.cc:455] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be04af34f168433e8d067a0a12725de8. Latest consensus state: current_term: 1 leader_uuid: "be04af34f168433e8d067a0a12725de8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be04af34f168433e8d067a0a12725de8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44127 } } }
I20260810 01:03:05.292987 11747 sys_catalog.cc:458] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:05.294752 11754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:03:05.300925 11754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:03:05.311091 11754 catalog_manager.cc:1347] Generated new cluster ID: 5410f646000e498496f47a8d008bd655
I20260810 01:03:05.311215 11754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:03:05.323165 11754 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:03:05.325163 11754 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:03:05.334309 11754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be04af34f168433e8d067a0a12725de8: Generated new TSK 0
I20260810 01:03:05.335391 11754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:03:05.345562 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 11657
W20260810 01:03:05.347066 11727 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57213
W20260810 01:03:05.347811 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 11657: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 01:03:05.348003 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 11657
W20260810 01:03:05.371553 11641 connection.cc:535] client connection to 127.0.0.1:44127 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44127): Connection reset by peer (error 104)
2026-08-10T01:03:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2129 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:03:05 dist-test-slave-gp6n krb5kdc[11768](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:03:05 dist-test-slave-gp6n krb5kdc[11768](info): set up 2 sockets
Aug 10 01:03:05 dist-test-slave-gp6n krb5kdc[11768](info): commencing operation
krb5kdc: starting...
W20260810 01:03:06.620771 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.004s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:03:06 dist-test-slave-gp6n krb5kdc[11768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:03:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:03:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 01:03:06.963107 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44827
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44827
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:03:07.322201 11783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:07.322539 11783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:07.322721 11783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:07.330260 11783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:07.330410 11783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:07.330533 11783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:07.330655 11783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:07.330726 11783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:07.341018 11783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44827
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:07.342422 11783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:07.343799 11783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:07.355955 11790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:07.355971 11792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:07.356544 11788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:07.357615 11783 server_base.cc:1034] running on GCE node
I20260810 01:03:07.358378 11783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:07.359871 11783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:07.361115 11783 hybrid_clock.cc:648] HybridClock initialized: now 1786323787361081 us; error 58 us; skew 500 ppm
I20260810 01:03:07.361475 11783 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:07 dist-test-slave-gp6n krb5kdc[11768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323787, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:07.364804 11783 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 01:03:07.366386 11783 webserver.cc:460] Webserver started at http://127.0.0.1:40297/ using document root <none> and password file <none>
I20260810 01:03:07.367434 11783 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:07.367573 11783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:07.367853 11783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:07.370886 11783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "7240e4cb9213401890232b1881db5f7e"
format_stamp: "Formatted at 2026-08-10 01:03:07 on dist-test-slave-gp6n"
server_key: "8c88ba37b55751d8cd69bce2dae37144"
server_key_iv: "a3dcb9db7b4203fb5a493a608e44d303"
server_key_version: "encryptionkey@0"
I20260810 01:03:07.371610 11783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "7240e4cb9213401890232b1881db5f7e"
format_stamp: "Formatted at 2026-08-10 01:03:07 on dist-test-slave-gp6n"
server_key: "8c88ba37b55751d8cd69bce2dae37144"
server_key_iv: "a3dcb9db7b4203fb5a493a608e44d303"
server_key_version: "encryptionkey@0"
I20260810 01:03:07.376780 11783 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 01:03:07.380221 11799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:07.381538 11783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 01:03:07.381722 11783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "7240e4cb9213401890232b1881db5f7e"
format_stamp: "Formatted at 2026-08-10 01:03:07 on dist-test-slave-gp6n"
server_key: "8c88ba37b55751d8cd69bce2dae37144"
server_key_iv: "a3dcb9db7b4203fb5a493a608e44d303"
server_key_version: "encryptionkey@0"
I20260810 01:03:07.381927 11783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:07.387820 11783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:07.391961 11783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:07.392257 11783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:07.411365 11783 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44827
I20260810 01:03:07.411428 11859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44827 every 8 connection(s)
I20260810 01:03:07.412856 11783 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:03:07.416584 11860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:07.417423 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 11783
I20260810 01:03:07.417639 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:03:07.418237 20806 external_mini_cluster.cc:1428] Setting key a6a2901d9f7d7bf2e74396c8f0c95b6e
I20260810 01:03:07.426481 11863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:07.420190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45059 (local address 127.0.0.1:44827)
0810 01:03:07.421007 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:07.421018 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:07.421062 (+ 44us) server_negotiation.cc:407] Connection header received
0810 01:03:07.423004 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:07.423025 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:07.423365 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:07.423697 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:07.424850 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:07.424857 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:07.424869 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:07.424885 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:07.424952 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:07.424977 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:07.425163 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:07.425275 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 01:03:07.425505 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":363,"threads_started":1}
I20260810 01:03:07.429788 11860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e: Bootstrap starting.
I20260810 01:03:07.430312 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 01:03:07.433625 11860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:07.435144 11860 log.cc:824] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:07.438781 11863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:07.436100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33805 (local address 127.0.0.1:44827)
0810 01:03:07.436311 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:07.436317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:07.436575 (+ 258us) server_negotiation.cc:407] Connection header received
0810 01:03:07.436800 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:07.436804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:07.436872 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:07.436957 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:07.438005 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:07.438010 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:07.438015 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:07.438034 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:07.438100 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:07.438129 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:07.438322 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:07.438379 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 01:03:07.438475 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 01:03:07.439464 11860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e: No bootstrap required, opened a new log
I20260810 01:03:07.447643 11860 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7240e4cb9213401890232b1881db5f7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44827 } }
I20260810 01:03:07.448014 11860 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:03:07.448097 11860 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7240e4cb9213401890232b1881db5f7e, State: Initialized, Role: FOLLOWER
I20260810 01:03:07.448686 11860 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7240e4cb9213401890232b1881db5f7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44827 } }
I20260810 01:03:07.448909 11860 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:03:07.449019 11860 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:03:07.449149 11860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:07.451694 11860 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7240e4cb9213401890232b1881db5f7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44827 } }
I20260810 01:03:07.452158 11860 leader_election.cc:302] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7240e4cb9213401890232b1881db5f7e; no voters:
I20260810 01:03:07.452709 11860 leader_election.cc:288] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:03:07.452878 11871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:03:07.453584 11871 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e [term 1 LEADER]: Becoming Leader. State: Replica: 7240e4cb9213401890232b1881db5f7e, State: Running, Role: LEADER
I20260810 01:03:07.454123 11871 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7240e4cb9213401890232b1881db5f7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44827 } }
I20260810 01:03:07.455195 11860 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:07.457367 11872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7240e4cb9213401890232b1881db5f7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7240e4cb9213401890232b1881db5f7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44827 } } }
I20260810 01:03:07.457901 11872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:07.458503 11873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7240e4cb9213401890232b1881db5f7e. Latest consensus state: current_term: 1 leader_uuid: "7240e4cb9213401890232b1881db5f7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7240e4cb9213401890232b1881db5f7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44827 } } }
I20260810 01:03:07.458837 11873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:07.467579 11881 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:03:07.473037 11881 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:03:07.482883 11881 catalog_manager.cc:1347] Generated new cluster ID: b80ebef6e28c4e34b044c7338b0b1668
I20260810 01:03:07.483034 11881 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:03:07.494362 11881 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:03:07.494814 11881 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:03:07.504367 11881 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7240e4cb9213401890232b1881db5f7e: Generated new TSK 0
I20260810 01:03:07.505390 11881 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:03:07.518105 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 11783
W20260810 01:03:07.519742 11854 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45059
W20260810 01:03:07.520445 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 11783: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 01:03:07.520628 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 11783
2026-08-10T01:03:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2168 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:03:07 dist-test-slave-gp6n krb5kdc[11896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:03:07 dist-test-slave-gp6n krb5kdc[11896](info): set up 2 sockets
Aug 10 01:03:07 dist-test-slave-gp6n krb5kdc[11896](info): commencing operation
krb5kdc: starting...
W20260810 01:03:08.717579 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.002s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:03:08 dist-test-slave-gp6n krb5kdc[11896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:03:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:03:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 01:03:09.053864 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41615
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:03:09.401134 11913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:09.401535 11913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:09.401700 11913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:09.408987 11913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:09.409157 11913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:09.409265 11913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:09.409386 11913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:09.409479 11913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:09.419438 11913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:09.420815 11913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:09.422165 11913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:09.434660 11919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:09.434623 11922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:09.434706 11920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:09.435662 11913 server_base.cc:1034] running on GCE node
I20260810 01:03:09.436513 11913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:09.438115 11913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:09.439353 11913 hybrid_clock.cc:648] HybridClock initialized: now 1786323789439280 us; error 70 us; skew 500 ppm
I20260810 01:03:09.439700 11913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:09 dist-test-slave-gp6n krb5kdc[11896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323789, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:09.447623 11913 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 01:03:09.449230 11913 webserver.cc:460] Webserver started at http://127.0.0.1:33433/ using document root <none> and password file <none>
I20260810 01:03:09.449913 11913 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:09.450021 11913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:09.450278 11913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:09.453576 11913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "dca8be7d72464c7e84930975cb337182"
format_stamp: "Formatted at 2026-08-10 01:03:09 on dist-test-slave-gp6n"
server_key: "29ac73eae21e6f8f8df9500230e340d0"
server_key_iv: "4bd6ebd473a6ec01d7a13498f6e95360"
server_key_version: "encryptionkey@0"
I20260810 01:03:09.454360 11913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "dca8be7d72464c7e84930975cb337182"
format_stamp: "Formatted at 2026-08-10 01:03:09 on dist-test-slave-gp6n"
server_key: "29ac73eae21e6f8f8df9500230e340d0"
server_key_iv: "4bd6ebd473a6ec01d7a13498f6e95360"
server_key_version: "encryptionkey@0"
I20260810 01:03:09.459950 11913 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 01:03:09.463382 11930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:09.464797 11913 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 01:03:09.464952 11913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "dca8be7d72464c7e84930975cb337182"
format_stamp: "Formatted at 2026-08-10 01:03:09 on dist-test-slave-gp6n"
server_key: "29ac73eae21e6f8f8df9500230e340d0"
server_key_iv: "4bd6ebd473a6ec01d7a13498f6e95360"
server_key_version: "encryptionkey@0"
I20260810 01:03:09.465123 11913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:09.480863 11913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:09.485240 11913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:09.485486 11913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:09.504885 11913 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41615
I20260810 01:03:09.504916 11986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41615 every 8 connection(s)
I20260810 01:03:09.506254 11913 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:03:09.506671 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 11913
I20260810 01:03:09.506845 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:03:09.507403 20806 external_mini_cluster.cc:1428] Setting key 038659c0c83445a5a7d37a281ac96afa
I20260810 01:03:09.510702 11987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:09.521262 11987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182: Bootstrap starting.
Aug 10 01:03:09 dist-test-slave-gp6n krb5kdc[11896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 01:03:09.524652 11987 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:09.525777 11987 log.cc:824] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:09.528620 11987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182: No bootstrap required, opened a new log
I20260810 01:03:09.530175 11990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:09.509487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49557 (local address 127.0.0.1:41615)
0810 01:03:09.510462 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:09.510478 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:09.510894 (+ 416us) server_negotiation.cc:407] Connection header received
0810 01:03:09.512703 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:09.512723 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:09.513097 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:09.513535 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:09.515092 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:09.516410 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:09.517725 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:09.518482 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:09.518686 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:09.522950 (+ 4264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:09.522959 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:09.522973 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:09.523011 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:09.526155 (+ 3144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:09.526939 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:09.526945 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:09.526951 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:09.527112 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:09.527522 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:09.527531 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:09.527533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:09.528033 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:09.528354 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:09.528784 (+ 430us) server_negotiation.cc:299] Negotiation successful
0810 01:03:09.529199 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":397,"threads_started":1}
I20260810 01:03:09.534310 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 01:03:09.537214 11987 raft_consensus.cc:348] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca8be7d72464c7e84930975cb337182" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41615 } }
I20260810 01:03:09.537583 11987 raft_consensus.cc:374] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:03:09.537707 11987 raft_consensus.cc:729] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dca8be7d72464c7e84930975cb337182, State: Initialized, Role: FOLLOWER
I20260810 01:03:09.538331 11987 consensus_queue.cc:260] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca8be7d72464c7e84930975cb337182" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41615 } }
I20260810 01:03:09.538641 11987 raft_consensus.cc:388] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:03:09.538825 11987 raft_consensus.cc:482] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:03:09.539037 11987 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:09.542295 11987 raft_consensus.cc:504] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca8be7d72464c7e84930975cb337182" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41615 } }
I20260810 01:03:09.542836 11987 leader_election.cc:302] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dca8be7d72464c7e84930975cb337182; no voters:
I20260810 01:03:09.543501 11987 leader_election.cc:288] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:03:09.543620 11999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:03:09.545006 11999 raft_consensus.cc:686] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182 [term 1 LEADER]: Becoming Leader. State: Replica: dca8be7d72464c7e84930975cb337182, State: Running, Role: LEADER
I20260810 01:03:09.546002 11987 sys_catalog.cc:564] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:09.545943 11999 consensus_queue.cc:237] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca8be7d72464c7e84930975cb337182" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41615 } }
I20260810 01:03:09.549225 12000 sys_catalog.cc:455] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dca8be7d72464c7e84930975cb337182" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca8be7d72464c7e84930975cb337182" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41615 } } }
I20260810 01:03:09.549602 12000 sys_catalog.cc:458] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:09.550202 12002 sys_catalog.cc:455] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dca8be7d72464c7e84930975cb337182. Latest consensus state: current_term: 1 leader_uuid: "dca8be7d72464c7e84930975cb337182" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca8be7d72464c7e84930975cb337182" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41615 } } }
I20260810 01:03:09.550644 12002 sys_catalog.cc:458] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:09.550814 12007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:03:09.556162 11990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:09.538926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48187 (local address 127.0.0.1:41615)
0810 01:03:09.539150 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:09.539156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:09.539551 (+ 395us) server_negotiation.cc:407] Connection header received
0810 01:03:09.539737 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:09.539741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:09.539799 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:09.539882 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:09.541710 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:09.542462 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:09.543778 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:09.544542 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:09.544658 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:09.547554 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:09.547561 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:09.547568 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:09.547611 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:09.551729 (+ 4118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:09.552718 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:09.552723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:09.552727 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:09.552906 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:09.553538 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:09.553545 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:09.553551 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:09.553932 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:09.554201 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:09.555491 (+ 1290us) server_negotiation.cc:299] Negotiation successful
0810 01:03:09.555755 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 01:03:09.558507 12007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:03:09.570297 12007 catalog_manager.cc:1347] Generated new cluster ID: 6652915a613b4c01af200b01551c5ee7
I20260810 01:03:09.570457 12007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:03:09.581601 12007 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:03:09.582908 12007 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:03:09.590879 12007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dca8be7d72464c7e84930975cb337182: Generated new TSK 0
I20260810 01:03:09.591730 12007 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:03:09.598207 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 11913
I20260810 01:03:09.599457 11982 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:03:09.690450 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 11913
2026-08-10T01:03:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2168 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:03:09 dist-test-slave-gp6n krb5kdc[12024](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:03:09 dist-test-slave-gp6n krb5kdc[12024](info): set up 2 sockets
Aug 10 01:03:09 dist-test-slave-gp6n krb5kdc[12024](info): commencing operation
krb5kdc: starting...
W20260810 01:03:10.947927 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.001s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:03:11 dist-test-slave-gp6n krb5kdc[12024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:03:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:03:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 01:03:11.286392 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34643
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:03:11.641134 12040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:11.641445 12040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:11.641613 12040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:11.649148 12040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:11.649264 12040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:11.649415 12040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:11.649533 12040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:11.649595 12040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:11.659916 12040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:11.661248 12040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:11.662638 12040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:11.675429 12046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:11.675551 12049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:11.675551 12047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:11.676550 12040 server_base.cc:1034] running on GCE node
I20260810 01:03:11.677278 12040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:11.678700 12040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:11.679935 12040 hybrid_clock.cc:648] HybridClock initialized: now 1786323791679889 us; error 34 us; skew 500 ppm
I20260810 01:03:11.680272 12040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:11 dist-test-slave-gp6n krb5kdc[12024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323791, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:11.683665 12040 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 01:03:11.685111 12040 webserver.cc:460] Webserver started at http://127.0.0.1:41193/ using document root <none> and password file <none>
I20260810 01:03:11.685793 12040 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:11.685947 12040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:11.686254 12040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:11.689254 12040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "73fa91a8688e4220a79d32211f4312e7"
format_stamp: "Formatted at 2026-08-10 01:03:11 on dist-test-slave-gp6n"
server_key: "f5bd1b6c3292d45b047ac876c1ef35d2"
server_key_iv: "d1f7d4ca9d79239bbd49bd39ac8c990e"
server_key_version: "encryptionkey@0"
I20260810 01:03:11.690045 12040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "73fa91a8688e4220a79d32211f4312e7"
format_stamp: "Formatted at 2026-08-10 01:03:11 on dist-test-slave-gp6n"
server_key: "f5bd1b6c3292d45b047ac876c1ef35d2"
server_key_iv: "d1f7d4ca9d79239bbd49bd39ac8c990e"
server_key_version: "encryptionkey@0"
I20260810 01:03:11.695178 12040 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 01:03:11.698586 12057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:11.699873 12040 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 01:03:11.700057 12040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "73fa91a8688e4220a79d32211f4312e7"
format_stamp: "Formatted at 2026-08-10 01:03:11 on dist-test-slave-gp6n"
server_key: "f5bd1b6c3292d45b047ac876c1ef35d2"
server_key_iv: "d1f7d4ca9d79239bbd49bd39ac8c990e"
server_key_version: "encryptionkey@0"
I20260810 01:03:11.700248 12040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:11.729204 12040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:11.733726 12040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:11.733973 12040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:11.753377 12040 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34643
I20260810 01:03:11.753422 12114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34643 every 8 connection(s)
I20260810 01:03:11.754846 12040 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:03:11.758751 12115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:11.760155 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 12040
I20260810 01:03:11.760351 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:03:11.760926 20806 external_mini_cluster.cc:1428] Setting key df97314618b8fe712e50e25cebc51ff8
I20260810 01:03:11.772905 12115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7: Bootstrap starting.
I20260810 01:03:11.774597 12120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:11.762818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52349 (local address 127.0.0.1:34643)
0810 01:03:11.763656 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:11.763667 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:11.763709 (+ 42us) server_negotiation.cc:407] Connection header received
0810 01:03:11.765308 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:11.765332 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:11.765730 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:11.766056 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:11.766717 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:11.767983 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:11.769910 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:11.770690 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:11.770816 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:11.772083 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:11.772091 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:11.772104 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:11.772661 (+ 557us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:11.772739 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:11.772764 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:11.772850 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:11.773161 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 01:03:11.773478 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":409,"threads_started":1}
I20260810 01:03:11.776309 12115 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:11.777509 12115 log.cc:824] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:11.778220 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 01:03:11.780347 12115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7: No bootstrap required, opened a new log
I20260810 01:03:11.789269 12120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:11.782755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38809 (local address 127.0.0.1:34643)
0810 01:03:11.782946 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:11.782961 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:11.783370 (+ 409us) server_negotiation.cc:407] Connection header received
0810 01:03:11.783508 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:11.783515 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:11.783594 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:11.783717 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:11.784200 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:11.785334 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:11.786666 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:11.787360 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:11.787507 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:11.788536 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:11.788542 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:11.788547 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:11.788568 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:11.788613 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:11.788634 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:11.788733 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:11.788858 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 01:03:11.788946 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 01:03:11.789822 12115 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73fa91a8688e4220a79d32211f4312e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34643 } }
I20260810 01:03:11.790217 12115 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:03:11.790340 12115 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73fa91a8688e4220a79d32211f4312e7, State: Initialized, Role: FOLLOWER
I20260810 01:03:11.791076 12115 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73fa91a8688e4220a79d32211f4312e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34643 } }
I20260810 01:03:11.791330 12115 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:03:11.791474 12115 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:03:11.791648 12115 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:11.793865 12115 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73fa91a8688e4220a79d32211f4312e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34643 } }
I20260810 01:03:11.794380 12115 leader_election.cc:302] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73fa91a8688e4220a79d32211f4312e7; no voters:
I20260810 01:03:11.795034 12115 leader_election.cc:288] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:03:11.795271 12127 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:03:11.796038 12127 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7 [term 1 LEADER]: Becoming Leader. State: Replica: 73fa91a8688e4220a79d32211f4312e7, State: Running, Role: LEADER
I20260810 01:03:11.796677 12127 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73fa91a8688e4220a79d32211f4312e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34643 } }
I20260810 01:03:11.797499 12115 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:11.799878 12129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73fa91a8688e4220a79d32211f4312e7. Latest consensus state: current_term: 1 leader_uuid: "73fa91a8688e4220a79d32211f4312e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73fa91a8688e4220a79d32211f4312e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34643 } } }
I20260810 01:03:11.800365 12129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:11.801169 12128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73fa91a8688e4220a79d32211f4312e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73fa91a8688e4220a79d32211f4312e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34643 } } }
I20260810 01:03:11.801476 12128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:11.803567 12138 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:03:11.808825 12138 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:03:11.818481 12138 catalog_manager.cc:1347] Generated new cluster ID: ba734cd33db04e2481853f1b7dce9bb9
I20260810 01:03:11.818596 12138 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:03:11.853657 12138 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:03:11.855628 12138 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:03:11.866816 12138 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73fa91a8688e4220a79d32211f4312e7: Generated new TSK 0
I20260810 01:03:11.867904 12138 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:03:11.874622 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 12040
W20260810 01:03:11.875718 12110 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52349
W20260810 01:03:11.876415 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 12040: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 01:03:11.876559 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 12040
2026-08-10T01:03:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2187 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:03:11 dist-test-slave-gp6n krb5kdc[12153](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:03:11 dist-test-slave-gp6n krb5kdc[12153](info): set up 2 sockets
Aug 10 01:03:11 dist-test-slave-gp6n krb5kdc[12153](info): commencing operation
krb5kdc: starting...
W20260810 01:03:13.076251 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:03:13 dist-test-slave-gp6n krb5kdc[12153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:03:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:03:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 01:03:13.410298 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45349
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:03:13.764308 12167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:13.764609 12167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:13.764787 12167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:13.774708 12167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:13.774855 12167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:13.774967 12167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:13.775072 12167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:13.775153 12167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:13.785370 12167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:13.786793 12167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:13.788165 12167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:13.800319 12173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:13.800463 12177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:13.800341 12175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:13.801239 12167 server_base.cc:1034] running on GCE node
I20260810 01:03:13.802038 12167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:13.803543 12167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:13.804774 12167 hybrid_clock.cc:648] HybridClock initialized: now 1786323793804727 us; error 27 us; skew 500 ppm
I20260810 01:03:13.805162 12167 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:13 dist-test-slave-gp6n krb5kdc[12153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323793, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:13.808679 12167 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 01:03:13.810185 12167 webserver.cc:460] Webserver started at http://127.0.0.1:37267/ using document root <none> and password file <none>
I20260810 01:03:13.810927 12167 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:13.811041 12167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:13.811306 12167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:13.814368 12167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "ccd370b2dd444d19972fe1241ee84d19"
format_stamp: "Formatted at 2026-08-10 01:03:13 on dist-test-slave-gp6n"
server_key: "db10c851a5e638474f0ca4a32b771e12"
server_key_iv: "e436345822c7c56352378cc01d6bba53"
server_key_version: "encryptionkey@0"
I20260810 01:03:13.815232 12167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "ccd370b2dd444d19972fe1241ee84d19"
format_stamp: "Formatted at 2026-08-10 01:03:13 on dist-test-slave-gp6n"
server_key: "db10c851a5e638474f0ca4a32b771e12"
server_key_iv: "e436345822c7c56352378cc01d6bba53"
server_key_version: "encryptionkey@0"
I20260810 01:03:13.820741 12167 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260810 01:03:13.824919 12186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:13.826534 12167 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 01:03:13.826725 12167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "ccd370b2dd444d19972fe1241ee84d19"
format_stamp: "Formatted at 2026-08-10 01:03:13 on dist-test-slave-gp6n"
server_key: "db10c851a5e638474f0ca4a32b771e12"
server_key_iv: "e436345822c7c56352378cc01d6bba53"
server_key_version: "encryptionkey@0"
I20260810 01:03:13.826938 12167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:13.832736 12167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:13.836922 12167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:13.837252 12167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:13.856381 12167 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45349
I20260810 01:03:13.856431 12244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45349 every 8 connection(s)
I20260810 01:03:13.857828 12167 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:03:13.861626 12246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:13.864729 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 12167
I20260810 01:03:13.864964 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:03:13.865435 20806 external_mini_cluster.cc:1428] Setting key f13ae27b8fcc126d65268e89015d3438
I20260810 01:03:13.874791 12250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:13.867501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57405 (local address 127.0.0.1:45349)
0810 01:03:13.868293 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:13.868309 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:13.868357 (+ 48us) server_negotiation.cc:407] Connection header received
0810 01:03:13.870760 (+ 2403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:13.870797 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:13.871285 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:13.871742 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:13.872785 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:13.872793 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:13.872810 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:13.872827 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:13.872899 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:13.872930 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:13.873175 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:13.873319 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 01:03:13.873509 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":329,"threads_started":1}
I20260810 01:03:13.875561 12246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19: Bootstrap starting.
I20260810 01:03:13.878789 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 01:03:13.879138 12246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:13.880398 12246 log.cc:824] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:13.883948 12246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19: No bootstrap required, opened a new log
I20260810 01:03:13.886689 12250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:13.884175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52163 (local address 127.0.0.1:45349)
0810 01:03:13.884388 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:13.884396 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:13.884766 (+ 370us) server_negotiation.cc:407] Connection header received
0810 01:03:13.884883 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:13.884888 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:13.884953 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:13.885058 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:13.886010 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:13.886021 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:13.886026 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:13.886049 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:13.886096 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:13.886121 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:13.886237 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:13.886309 (+ 72us) server_negotiation.cc:299] Negotiation successful
0810 01:03:13.886371 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 01:03:13.893113 12246 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd370b2dd444d19972fe1241ee84d19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45349 } }
I20260810 01:03:13.893463 12246 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:03:13.893535 12246 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccd370b2dd444d19972fe1241ee84d19, State: Initialized, Role: FOLLOWER
I20260810 01:03:13.894200 12246 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd370b2dd444d19972fe1241ee84d19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45349 } }
I20260810 01:03:13.894544 12246 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:03:13.894706 12246 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:03:13.894840 12246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:13.896991 12246 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd370b2dd444d19972fe1241ee84d19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45349 } }
I20260810 01:03:13.897430 12246 leader_election.cc:302] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccd370b2dd444d19972fe1241ee84d19; no voters:
I20260810 01:03:13.897974 12246 leader_election.cc:288] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:03:13.898140 12257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:03:13.898873 12257 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19 [term 1 LEADER]: Becoming Leader. State: Replica: ccd370b2dd444d19972fe1241ee84d19, State: Running, Role: LEADER
I20260810 01:03:13.899452 12257 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd370b2dd444d19972fe1241ee84d19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45349 } }
I20260810 01:03:13.900202 12246 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:13.902328 12259 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccd370b2dd444d19972fe1241ee84d19. Latest consensus state: current_term: 1 leader_uuid: "ccd370b2dd444d19972fe1241ee84d19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd370b2dd444d19972fe1241ee84d19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45349 } } }
I20260810 01:03:13.902755 12259 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:13.903293 12258 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccd370b2dd444d19972fe1241ee84d19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd370b2dd444d19972fe1241ee84d19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45349 } } }
I20260810 01:03:13.903653 12258 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:13.907400 12263 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:03:13.913272 12263 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:03:13.922874 12263 catalog_manager.cc:1347] Generated new cluster ID: 047b58ab07ff491091754abc8970847b
I20260810 01:03:13.923005 12263 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:03:13.943688 12263 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:03:13.944156 12263 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:03:13.963233 12263 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccd370b2dd444d19972fe1241ee84d19: Generated new TSK 0
I20260810 01:03:13.964140 12263 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:03:13.971526 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 12167
W20260810 01:03:13.972679 12239 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57405
W20260810 01:03:13.973405 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 12167: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 01:03:13.973563 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 12167
2026-08-10T01:03:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2099 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:03:14 dist-test-slave-gp6n krb5kdc[12280](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:03:14 dist-test-slave-gp6n krb5kdc[12280](info): set up 2 sockets
Aug 10 01:03:14 dist-test-slave-gp6n krb5kdc[12280](info): commencing operation
krb5kdc: starting...
W20260810 01:03:15.271941 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.005s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:03:15 dist-test-slave-gp6n krb5kdc[12280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:03:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:03:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 01:03:15.642388 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35617
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:03:15.989008 12295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:15.989401 12295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:15.989610 12295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:15.996790 12295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:15.996946 12295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:15.997058 12295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:15.997177 12295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:15.997265 12295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:16.007062 12295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:16.008473 12295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:16.009855 12295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:16.022478 12301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:16.022435 12304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:16.022981 12300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:16.023317 12295 server_base.cc:1034] running on GCE node
I20260810 01:03:16.024137 12295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:16.025574 12295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:16.026810 12295 hybrid_clock.cc:648] HybridClock initialized: now 1786323796026770 us; error 51 us; skew 500 ppm
I20260810 01:03:16.027180 12295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:16 dist-test-slave-gp6n krb5kdc[12280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323796, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:16.030481 12295 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 01:03:16.031988 12295 webserver.cc:460] Webserver started at http://127.0.0.1:33629/ using document root <none> and password file <none>
I20260810 01:03:16.032629 12295 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:16.032732 12295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:16.032987 12295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:16.036085 12295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "83ac5c8f00244b7989e3ac976d3858ee"
format_stamp: "Formatted at 2026-08-10 01:03:16 on dist-test-slave-gp6n"
server_key: "204c48714e30e3e0416a2b655b3bef65"
server_key_iv: "3fdf495fc1bb685b462a688487864c65"
server_key_version: "encryptionkey@0"
I20260810 01:03:16.036828 12295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "83ac5c8f00244b7989e3ac976d3858ee"
format_stamp: "Formatted at 2026-08-10 01:03:16 on dist-test-slave-gp6n"
server_key: "204c48714e30e3e0416a2b655b3bef65"
server_key_iv: "3fdf495fc1bb685b462a688487864c65"
server_key_version: "encryptionkey@0"
I20260810 01:03:16.041976 12295 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 01:03:16.045521 12311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:16.046896 12295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:03:16.047084 12295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "83ac5c8f00244b7989e3ac976d3858ee"
format_stamp: "Formatted at 2026-08-10 01:03:16 on dist-test-slave-gp6n"
server_key: "204c48714e30e3e0416a2b655b3bef65"
server_key_iv: "3fdf495fc1bb685b462a688487864c65"
server_key_version: "encryptionkey@0"
I20260810 01:03:16.047257 12295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:16.074209 12295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:16.079233 12295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:16.079564 12295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:16.099152 12295 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35617
I20260810 01:03:16.099200 12370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35617 every 8 connection(s)
I20260810 01:03:16.100595 12295 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:03:16.104314 12371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:16.106738 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 12295
I20260810 01:03:16.106935 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:03:16.107496 20806 external_mini_cluster.cc:1428] Setting key 0a66625b641ac9ca6b40014f7111c54f
I20260810 01:03:16.117046 12371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee: Bootstrap starting.
I20260810 01:03:16.121091 12371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:16.122557 12371 log.cc:824] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee: Log is configured to *not* fsync() on all Append() calls
Aug 10 01:03:16 dist-test-slave-gp6n krb5kdc[12280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 01:03:16.125571 12371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee: No bootstrap required, opened a new log
I20260810 01:03:16.132701 12374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:16.109501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:39511 (local address 127.0.0.1:35617)
0810 01:03:16.110354 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:16.110366 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:16.110452 (+ 86us) server_negotiation.cc:407] Connection header received
0810 01:03:16.112139 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:16.112162 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:16.112558 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:16.112879 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:16.114756 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:16.116154 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:16.117992 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:16.118814 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:16.118962 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:16.124229 (+ 5267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:16.124237 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:16.124252 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:16.124305 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:16.127990 (+ 3685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:16.128962 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:16.128974 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:16.128980 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:16.129164 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:16.129727 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:16.129731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:16.129733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:16.130313 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:16.130724 (+ 411us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:16.131254 (+ 530us) server_negotiation.cc:299] Negotiation successful
0810 01:03:16.131598 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":405,"threads_started":1}
I20260810 01:03:16.133754 12371 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ac5c8f00244b7989e3ac976d3858ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35617 } }
I20260810 01:03:16.134150 12371 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:03:16.134295 12371 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ac5c8f00244b7989e3ac976d3858ee, State: Initialized, Role: FOLLOWER
I20260810 01:03:16.134871 12371 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ac5c8f00244b7989e3ac976d3858ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35617 } }
I20260810 01:03:16.135138 12371 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:03:16.135281 12371 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:03:16.135501 12371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:16.137066 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 01:03:16.138089 12371 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ac5c8f00244b7989e3ac976d3858ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35617 } }
I20260810 01:03:16.138684 12371 leader_election.cc:302] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83ac5c8f00244b7989e3ac976d3858ee; no voters:
I20260810 01:03:16.139313 12371 leader_election.cc:288] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:03:16.139528 12376 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:03:16.141264 12371 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:16.147449 12376 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee [term 1 LEADER]: Becoming Leader. State: Replica: 83ac5c8f00244b7989e3ac976d3858ee, State: Running, Role: LEADER
I20260810 01:03:16.148340 12376 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ac5c8f00244b7989e3ac976d3858ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35617 } }
I20260810 01:03:16.152910 12377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83ac5c8f00244b7989e3ac976d3858ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ac5c8f00244b7989e3ac976d3858ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35617 } } }
I20260810 01:03:16.153488 12377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:16.155236 12379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83ac5c8f00244b7989e3ac976d3858ee. Latest consensus state: current_term: 1 leader_uuid: "83ac5c8f00244b7989e3ac976d3858ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ac5c8f00244b7989e3ac976d3858ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35617 } } }
I20260810 01:03:16.155869 12379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:16.165346 12397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:03:16.171778 12397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:03:16.174271 12374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:16.150473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:58517 (local address 127.0.0.1:35617)
0810 01:03:16.150730 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:16.150739 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:16.155425 (+ 4686us) server_negotiation.cc:407] Connection header received
0810 01:03:16.155773 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:16.155783 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:16.155896 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:16.155999 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:16.158805 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:16.160539 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:16.164093 (+ 3554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:16.164834 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:16.164977 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:16.168072 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:16.168077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:16.168080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:16.168112 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:16.171202 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:16.172125 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:16.172131 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:16.172133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:16.172256 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:16.172754 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:16.172758 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:16.172760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:16.173076 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:16.173360 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:16.173758 (+ 398us) server_negotiation.cc:299] Negotiation successful
0810 01:03:16.173946 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 01:03:16.182204 12397 catalog_manager.cc:1347] Generated new cluster ID: 196a757120464e19b05d77933e3ddd5b
I20260810 01:03:16.182329 12397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:03:16.205374 12397 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:03:16.206777 12397 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:03:16.216331 12397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83ac5c8f00244b7989e3ac976d3858ee: Generated new TSK 0
I20260810 01:03:16.217480 12397 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:03:16.225293 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 12295
I20260810 01:03:16.226384 12365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:03:16.319248 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 12295
W20260810 01:03:16.341933 12278 connection.cc:535] client connection to 127.0.0.1:35617 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35617): Connection reset by peer (error 104)
2026-08-10T01:03:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2344 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:03:16 dist-test-slave-gp6n krb5kdc[12407](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:03:16 dist-test-slave-gp6n krb5kdc[12407](info): set up 2 sockets
Aug 10 01:03:16 dist-test-slave-gp6n krb5kdc[12407](info): commencing operation
krb5kdc: starting...
W20260810 01:03:17.540884 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.003s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:03:17 dist-test-slave-gp6n krb5kdc[12407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:03:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:03:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 01:03:17.883766 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44475
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:03:18.246240 12420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:18.246618 12420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:18.246807 12420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:18.254081 12420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:18.254195 12420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:18.254300 12420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:18.254429 12420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:18.254505 12420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:18.264462 12420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:18.265772 12420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:18.267176 12420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:18.279623 12428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:18.279593 12429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:18.279557 12431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:18.280498 12420 server_base.cc:1034] running on GCE node
I20260810 01:03:18.281363 12420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:18.282912 12420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:18.284144 12420 hybrid_clock.cc:648] HybridClock initialized: now 1786323798284085 us; error 53 us; skew 500 ppm
I20260810 01:03:18.284520 12420 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:18 dist-test-slave-gp6n krb5kdc[12407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323798, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:18.287876 12420 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 01:03:18.289371 12420 webserver.cc:460] Webserver started at http://127.0.0.1:45007/ using document root <none> and password file <none>
I20260810 01:03:18.290069 12420 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:18.290215 12420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:18.290616 12420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:18.293624 12420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "57973298ff0b41fba40b8dea8366f58c"
format_stamp: "Formatted at 2026-08-10 01:03:18 on dist-test-slave-gp6n"
server_key: "912d301fc56126939422fc645aa580f5"
server_key_iv: "e34caa690497e572cdf03cc9dac6aa01"
server_key_version: "encryptionkey@0"
I20260810 01:03:18.294469 12420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "57973298ff0b41fba40b8dea8366f58c"
format_stamp: "Formatted at 2026-08-10 01:03:18 on dist-test-slave-gp6n"
server_key: "912d301fc56126939422fc645aa580f5"
server_key_iv: "e34caa690497e572cdf03cc9dac6aa01"
server_key_version: "encryptionkey@0"
I20260810 01:03:18.300580 12420 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 01:03:18.304260 12440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:18.305661 12420 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 01:03:18.305852 12420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "57973298ff0b41fba40b8dea8366f58c"
format_stamp: "Formatted at 2026-08-10 01:03:18 on dist-test-slave-gp6n"
server_key: "912d301fc56126939422fc645aa580f5"
server_key_iv: "e34caa690497e572cdf03cc9dac6aa01"
server_key_version: "encryptionkey@0"
I20260810 01:03:18.306066 12420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:18.332252 12420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:18.336818 12420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:18.337123 12420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:18.356493 12420 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44475
I20260810 01:03:18.356528 12494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44475 every 8 connection(s)
I20260810 01:03:18.357913 12420 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:03:18.361719 12495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:18.367226 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 12420
I20260810 01:03:18.367408 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:03:18.367952 20806 external_mini_cluster.cc:1428] Setting key bb071a35ef4b0cb9be08d64e708faadf
I20260810 01:03:18.373199 12495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c: Bootstrap starting.
I20260810 01:03:18.376458 12495 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:18.377863 12495 log.cc:824] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:18.381650 12495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c: No bootstrap required, opened a new log
I20260810 01:03:18.382341 12498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:18.369940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:55857 (local address 127.0.0.1:44475)
0810 01:03:18.370784 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:18.370796 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:18.370841 (+ 45us) server_negotiation.cc:407] Connection header received
0810 01:03:18.372516 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:18.372540 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:18.372931 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:18.373321 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:18.373873 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:18.375308 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:18.376696 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:18.377822 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:18.378053 (+ 231us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:18.379483 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:18.379495 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:18.379515 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:18.380242 (+ 727us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:18.380348 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:18.380392 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:18.380560 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:18.380817 (+ 257us) server_negotiation.cc:299] Negotiation successful
0810 01:03:18.381009 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":350,"threads_started":1}
I20260810 01:03:18.386292 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 01:03:18.391714 12495 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57973298ff0b41fba40b8dea8366f58c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44475 } }
I20260810 01:03:18.392297 12495 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:03:18.392431 12495 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57973298ff0b41fba40b8dea8366f58c, State: Initialized, Role: FOLLOWER
I20260810 01:03:18.393098 12495 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57973298ff0b41fba40b8dea8366f58c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44475 } }
I20260810 01:03:18.393342 12495 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:03:18.393522 12495 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:03:18.393715 12495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:18.396332 12495 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57973298ff0b41fba40b8dea8366f58c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44475 } }
I20260810 01:03:18.396872 12495 leader_election.cc:302] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57973298ff0b41fba40b8dea8366f58c; no voters:
I20260810 01:03:18.397578 12495 leader_election.cc:288] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:03:18.397771 12505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:03:18.398991 12505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c [term 1 LEADER]: Becoming Leader. State: Replica: 57973298ff0b41fba40b8dea8366f58c, State: Running, Role: LEADER
I20260810 01:03:18.399224 12498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:18.391904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:58837 (local address 127.0.0.1:44475)
0810 01:03:18.392094 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:18.392101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:18.392396 (+ 295us) server_negotiation.cc:407] Connection header received
0810 01:03:18.392648 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:18.392652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:18.392718 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:18.392823 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:18.393379 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:18.394307 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:18.395851 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:18.396344 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:18.396491 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:18.397743 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:18.397749 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:18.397752 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:18.397775 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:18.397815 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:18.397831 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:18.397983 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:18.398603 (+ 620us) server_negotiation.cc:299] Negotiation successful
0810 01:03:18.398675 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 01:03:18.399663 12505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57973298ff0b41fba40b8dea8366f58c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44475 } }
I20260810 01:03:18.400264 12495 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:18.402778 12508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57973298ff0b41fba40b8dea8366f58c. Latest consensus state: current_term: 1 leader_uuid: "57973298ff0b41fba40b8dea8366f58c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57973298ff0b41fba40b8dea8366f58c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44475 } } }
I20260810 01:03:18.402865 12506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57973298ff0b41fba40b8dea8366f58c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57973298ff0b41fba40b8dea8366f58c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44475 } } }
I20260810 01:03:18.403327 12506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:18.403203 12508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:18.405437 12514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:03:18.410898 12514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:03:18.419857 12514 catalog_manager.cc:1347] Generated new cluster ID: 44fedbaad233432e9b89b1462e594196
I20260810 01:03:18.420326 12514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:03:18.471961 12514 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:03:18.473380 12514 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:03:18.480890 12514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57973298ff0b41fba40b8dea8366f58c: Generated new TSK 0
I20260810 01:03:18.481737 12514 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:03:18.498574 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 12420
W20260810 01:03:18.499768 12490 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.44:55857
W20260810 01:03:18.500593 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 12420: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 01:03:18.500768 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 12420
W20260810 01:03:18.523707 12404 connection.cc:535] client connection to 127.0.0.1:44475 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44475): Connection reset by peer (error 104)
2026-08-10T01:03:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2181 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:03:18 dist-test-slave-gp6n krb5kdc[12532](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:03:18 dist-test-slave-gp6n krb5kdc[12532](info): set up 2 sockets
Aug 10 01:03:18 dist-test-slave-gp6n krb5kdc[12532](info): commencing operation
krb5kdc: starting...
W20260810 01:03:19.745512 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.001s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:03:19 dist-test-slave-gp6n krb5kdc[12532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:03:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:03:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 01:03:20.094072 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34363
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:03:20.442926 12547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:20.443264 12547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:20.443418 12547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:20.450574 12547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:20.450716 12547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:20.450821 12547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:20.450924 12547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:20.451004 12547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:20.460775 12547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:20.462129 12547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:20.463503 12547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:20.475920 12553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:20.475927 12554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:20.475945 12556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:20.476645 12547 server_base.cc:1034] running on GCE node
I20260810 01:03:20.477514 12547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:20.479063 12547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:20.480269 12547 hybrid_clock.cc:648] HybridClock initialized: now 1786323800480209 us; error 49 us; skew 500 ppm
I20260810 01:03:20.480659 12547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:20 dist-test-slave-gp6n krb5kdc[12532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323800, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:20.483939 12547 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 01:03:20.485457 12547 webserver.cc:460] Webserver started at http://127.0.0.1:40657/ using document root <none> and password file <none>
I20260810 01:03:20.486259 12547 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:20.486380 12547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:20.486714 12547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:20.489763 12547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "9b094eddc59e4dd28ad31050b1affeac"
format_stamp: "Formatted at 2026-08-10 01:03:20 on dist-test-slave-gp6n"
server_key: "e9685b3b028510dc59a877626acbce01"
server_key_iv: "64b063bd0a8a6535a078cae57b3da720"
server_key_version: "encryptionkey@0"
I20260810 01:03:20.490597 12547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "9b094eddc59e4dd28ad31050b1affeac"
format_stamp: "Formatted at 2026-08-10 01:03:20 on dist-test-slave-gp6n"
server_key: "e9685b3b028510dc59a877626acbce01"
server_key_iv: "64b063bd0a8a6535a078cae57b3da720"
server_key_version: "encryptionkey@0"
I20260810 01:03:20.495837 12547 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260810 01:03:20.499398 12565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:20.500860 12547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 01:03:20.501071 12547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "9b094eddc59e4dd28ad31050b1affeac"
format_stamp: "Formatted at 2026-08-10 01:03:20 on dist-test-slave-gp6n"
server_key: "e9685b3b028510dc59a877626acbce01"
server_key_iv: "64b063bd0a8a6535a078cae57b3da720"
server_key_version: "encryptionkey@0"
I20260810 01:03:20.501328 12547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:20.507380 12547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:20.516537 12547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:20.516834 12547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:20.536283 12547 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34363
I20260810 01:03:20.536379 12622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34363 every 8 connection(s)
I20260810 01:03:20.537837 12547 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:03:20.541683 12624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:20.547329 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 12547
I20260810 01:03:20.547528 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:03:20.548094 20806 external_mini_cluster.cc:1428] Setting key c342711128af3af673825d4840e1e42b
I20260810 01:03:20.552903 12624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac: Bootstrap starting.
I20260810 01:03:20.555954 12629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:20.549957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:47757 (local address 127.0.0.1:34363)
0810 01:03:20.550844 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:20.550856 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:20.550900 (+ 44us) server_negotiation.cc:407] Connection header received
0810 01:03:20.552524 (+ 1624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:20.552549 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:20.552881 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:20.553260 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:20.554327 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:20.554334 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:20.554347 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:20.554366 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:20.554478 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:20.554501 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:20.554678 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:20.554825 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 01:03:20.555064 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":440,"threads_started":1}
I20260810 01:03:20.556654 12624 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:20.557703 12624 log.cc:824] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:20.559597 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 01:03:20.560622 12624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac: No bootstrap required, opened a new log
I20260810 01:03:20.566562 12629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:20.564034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:52917 (local address 127.0.0.1:34363)
0810 01:03:20.564233 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:20.564238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:20.564552 (+ 314us) server_negotiation.cc:407] Connection header received
0810 01:03:20.564712 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:20.564716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:20.564762 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:20.564848 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:20.565872 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:20.565876 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:20.565879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:20.565897 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:20.565949 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:20.565964 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:20.566058 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:20.566188 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 01:03:20.566236 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 01:03:20.569082 12624 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b094eddc59e4dd28ad31050b1affeac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34363 } }
I20260810 01:03:20.569481 12624 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:03:20.569600 12624 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b094eddc59e4dd28ad31050b1affeac, State: Initialized, Role: FOLLOWER
I20260810 01:03:20.570164 12624 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b094eddc59e4dd28ad31050b1affeac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34363 } }
I20260810 01:03:20.570461 12624 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:03:20.570612 12624 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:03:20.570794 12624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:20.573046 12624 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b094eddc59e4dd28ad31050b1affeac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34363 } }
I20260810 01:03:20.573596 12624 leader_election.cc:302] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b094eddc59e4dd28ad31050b1affeac; no voters:
I20260810 01:03:20.574198 12624 leader_election.cc:288] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:03:20.574326 12636 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:03:20.575064 12636 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac [term 1 LEADER]: Becoming Leader. State: Replica: 9b094eddc59e4dd28ad31050b1affeac, State: Running, Role: LEADER
I20260810 01:03:20.575682 12636 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b094eddc59e4dd28ad31050b1affeac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34363 } }
I20260810 01:03:20.576383 12624 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:20.578578 12637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b094eddc59e4dd28ad31050b1affeac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b094eddc59e4dd28ad31050b1affeac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34363 } } }
I20260810 01:03:20.578670 12638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b094eddc59e4dd28ad31050b1affeac. Latest consensus state: current_term: 1 leader_uuid: "9b094eddc59e4dd28ad31050b1affeac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b094eddc59e4dd28ad31050b1affeac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34363 } } }
I20260810 01:03:20.579089 12638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:20.578974 12637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:20.580787 12643 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:03:20.586581 12643 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:03:20.596542 12643 catalog_manager.cc:1347] Generated new cluster ID: 7dc59c2403c940fe914780eb61f0d027
I20260810 01:03:20.596668 12643 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:03:20.616709 12643 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:03:20.617170 12643 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:03:20.624177 12643 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b094eddc59e4dd28ad31050b1affeac: Generated new TSK 0
I20260810 01:03:20.625038 12643 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:03:20.636394 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 12547
W20260810 01:03:20.637550 12618 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.44:47757
W20260810 01:03:20.638280 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 12547: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 01:03:20.638453 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 12547
2026-08-10T01:03:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2137 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:03:20 dist-test-slave-gp6n krb5kdc[12659](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:03:20 dist-test-slave-gp6n krb5kdc[12659](info): set up 2 sockets
Aug 10 01:03:20 dist-test-slave-gp6n krb5kdc[12659](info): commencing operation
krb5kdc: starting...
W20260810 01:03:21.850512 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.005s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:03:22 dist-test-slave-gp6n krb5kdc[12659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:03:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:03:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 01:03:22.198135 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42163
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:03:22.555310 12676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:22.555630 12676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:22.555792 12676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:22.563949 12676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:22.564057 12676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:22.564124 12676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:22.564213 12676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:22.564286 12676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:22.574064 12676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42163
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:22.575393 12676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:22.576707 12676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:22.588424 12681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:22.588383 12682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:22.588452 12684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:22.589178 12676 server_base.cc:1034] running on GCE node
I20260810 01:03:22.590078 12676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:22.591478 12676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:22.592703 12676 hybrid_clock.cc:648] HybridClock initialized: now 1786323802592664 us; error 46 us; skew 500 ppm
I20260810 01:03:22.593050 12676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:22 dist-test-slave-gp6n krb5kdc[12659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323802, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:22.596217 12676 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 01:03:22.597652 12676 webserver.cc:460] Webserver started at http://127.0.0.1:37219/ using document root <none> and password file <none>
I20260810 01:03:22.598295 12676 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:22.598477 12676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:22.598776 12676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:22.601706 12676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "a9f3f881103245409bce1ff22bc7c887"
format_stamp: "Formatted at 2026-08-10 01:03:22 on dist-test-slave-gp6n"
server_key: "966f6a3076bce027a04adc6a8d7c9f5e"
server_key_iv: "74e5c4e8a8531a87710e090e28530c18"
server_key_version: "encryptionkey@0"
I20260810 01:03:22.602537 12676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "a9f3f881103245409bce1ff22bc7c887"
format_stamp: "Formatted at 2026-08-10 01:03:22 on dist-test-slave-gp6n"
server_key: "966f6a3076bce027a04adc6a8d7c9f5e"
server_key_iv: "74e5c4e8a8531a87710e090e28530c18"
server_key_version: "encryptionkey@0"
I20260810 01:03:22.607617 12676 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 01:03:22.611016 12694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:22.612264 12676 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 01:03:22.612457 12676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "a9f3f881103245409bce1ff22bc7c887"
format_stamp: "Formatted at 2026-08-10 01:03:22 on dist-test-slave-gp6n"
server_key: "966f6a3076bce027a04adc6a8d7c9f5e"
server_key_iv: "74e5c4e8a8531a87710e090e28530c18"
server_key_version: "encryptionkey@0"
I20260810 01:03:22.612655 12676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:22.629570 12676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:22.633651 12676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:22.633952 12676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:22.652557 12676 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42163
I20260810 01:03:22.652592 12757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42163 every 8 connection(s)
I20260810 01:03:22.653961 12676 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:03:22.655857 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 12676
I20260810 01:03:22.656061 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:03:22.656644 20806 external_mini_cluster.cc:1428] Setting key bc45401a5c96ca0d8a60f640a756b574
I20260810 01:03:22.658382 12758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:22.669584 12758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887: Bootstrap starting.
Aug 10 01:03:22 dist-test-slave-gp6n krb5kdc[12659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 01:03:22.673033 12758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:22.674104 12758 log.cc:824] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:22.676921 12758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887: No bootstrap required, opened a new log
I20260810 01:03:22.678759 12763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:22.658555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:42111 (local address 127.0.0.1:42163)
0810 01:03:22.659321 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:22.659333 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:22.659371 (+ 38us) server_negotiation.cc:407] Connection header received
0810 01:03:22.660963 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:22.660983 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:22.661368 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:22.661754 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:22.663255 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:22.664572 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:22.665873 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:22.666642 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:22.666833 (+ 191us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:22.671289 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:22.671297 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:22.671310 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:22.671355 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:22.674465 (+ 3110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:22.675240 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:22.675253 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:22.675259 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:22.675437 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:22.675891 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:22.675896 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:22.675900 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:22.676403 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:22.676744 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:22.677237 (+ 493us) server_negotiation.cc:299] Negotiation successful
0810 01:03:22.677634 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":334,"threads_started":1}
I20260810 01:03:22.683761 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 01:03:22.686028 12758 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9f3f881103245409bce1ff22bc7c887" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42163 } }
I20260810 01:03:22.686450 12758 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:03:22.686568 12758 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9f3f881103245409bce1ff22bc7c887, State: Initialized, Role: FOLLOWER
I20260810 01:03:22.687134 12758 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9f3f881103245409bce1ff22bc7c887" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42163 } }
I20260810 01:03:22.687364 12758 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:03:22.687511 12758 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:03:22.687686 12758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:22.689807 12758 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9f3f881103245409bce1ff22bc7c887" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42163 } }
I20260810 01:03:22.690291 12758 leader_election.cc:302] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9f3f881103245409bce1ff22bc7c887; no voters:
I20260810 01:03:22.690999 12758 leader_election.cc:288] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:03:22.691128 12769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:03:22.693482 12758 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:22.699680 12769 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887 [term 1 LEADER]: Becoming Leader. State: Replica: a9f3f881103245409bce1ff22bc7c887, State: Running, Role: LEADER
I20260810 01:03:22.700361 12769 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9f3f881103245409bce1ff22bc7c887" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42163 } }
I20260810 01:03:22.704183 12771 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9f3f881103245409bce1ff22bc7c887" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9f3f881103245409bce1ff22bc7c887" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42163 } } }
I20260810 01:03:22.704173 12772 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9f3f881103245409bce1ff22bc7c887. Latest consensus state: current_term: 1 leader_uuid: "a9f3f881103245409bce1ff22bc7c887" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9f3f881103245409bce1ff22bc7c887" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42163 } } }
I20260810 01:03:22.704685 12771 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:22.704685 12772 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:22.720043 12783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:03:22.722325 12763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:22.692241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:50429 (local address 127.0.0.1:42163)
0810 01:03:22.692461 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:22.692470 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:22.699155 (+ 6685us) server_negotiation.cc:407] Connection header received
0810 01:03:22.699416 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:22.699424 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:22.699551 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:22.699705 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:22.701590 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:22.703299 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:22.705422 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:22.706365 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:22.706541 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:22.711049 (+ 4508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:22.711059 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:22.711066 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:22.711119 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:22.716158 (+ 5039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:22.717815 (+ 1657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:22.717823 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:22.717830 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:22.718154 (+ 324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:22.718969 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:22.718977 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:22.718983 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:22.719510 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:22.719881 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:22.721525 (+ 1644us) server_negotiation.cc:299] Negotiation successful
0810 01:03:22.721791 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 01:03:22.726164 12783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:03:22.735896 12783 catalog_manager.cc:1347] Generated new cluster ID: 8c2e28b7da4c4caa93b9b2fc9b061a56
I20260810 01:03:22.736037 12783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:03:22.750561 12783 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:03:22.752367 12783 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:03:22.766527 12783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9f3f881103245409bce1ff22bc7c887: Generated new TSK 0
I20260810 01:03:22.767690 12783 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:03:22.776485 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 12676
I20260810 01:03:22.778183 12753 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:03:22.869438 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 12676
W20260810 01:03:22.890661 12656 connection.cc:535] client connection to 127.0.0.1:42163 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42163): Connection reset by peer (error 104)
2026-08-10T01:03:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2234 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:03:22 dist-test-slave-gp6n krb5kdc[12798](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:03:22 dist-test-slave-gp6n krb5kdc[12798](info): set up 2 sockets
Aug 10 01:03:22 dist-test-slave-gp6n krb5kdc[12798](info): commencing operation
krb5kdc: starting...
W20260810 01:03:24.086568 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.003s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:03:24 dist-test-slave-gp6n krb5kdc[12798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:03:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:03:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 01:03:24.430228 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37717
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:03:24.776875 12812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:24.777176 12812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:24.777354 12812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:24.784647 12812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:24.784757 12812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:24.784862 12812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:24.784976 12812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:24.785051 12812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:24.795464 12812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37717
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:24.796761 12812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:24.798118 12812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:24.810035 12817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:24.810060 12820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:24.810202 12818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:24.811276 12812 server_base.cc:1034] running on GCE node
I20260810 01:03:24.812161 12812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:24.813493 12812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:24.814713 12812 hybrid_clock.cc:648] HybridClock initialized: now 1786323804814660 us; error 47 us; skew 500 ppm
I20260810 01:03:24.815063 12812 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:24 dist-test-slave-gp6n krb5kdc[12798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323804, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:24.818215 12812 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 01:03:24.819720 12812 webserver.cc:460] Webserver started at http://127.0.0.1:42931/ using document root <none> and password file <none>
I20260810 01:03:24.820346 12812 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:24.820462 12812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:24.820708 12812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:24.823787 12812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "a8d7c696e2db4c42b77a9eaf7cebc6e2"
format_stamp: "Formatted at 2026-08-10 01:03:24 on dist-test-slave-gp6n"
server_key: "f6ba85fe2ea074a45b86da67411caaa4"
server_key_iv: "c25f22b9f65e7244ea6dcf7079859ab9"
server_key_version: "encryptionkey@0"
I20260810 01:03:24.824502 12812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "a8d7c696e2db4c42b77a9eaf7cebc6e2"
format_stamp: "Formatted at 2026-08-10 01:03:24 on dist-test-slave-gp6n"
server_key: "f6ba85fe2ea074a45b86da67411caaa4"
server_key_iv: "c25f22b9f65e7244ea6dcf7079859ab9"
server_key_version: "encryptionkey@0"
I20260810 01:03:24.829582 12812 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260810 01:03:24.832957 12828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:24.834295 12812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 01:03:24.834522 12812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "a8d7c696e2db4c42b77a9eaf7cebc6e2"
format_stamp: "Formatted at 2026-08-10 01:03:24 on dist-test-slave-gp6n"
server_key: "f6ba85fe2ea074a45b86da67411caaa4"
server_key_iv: "c25f22b9f65e7244ea6dcf7079859ab9"
server_key_version: "encryptionkey@0"
I20260810 01:03:24.834690 12812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:24.880252 12812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:24.884833 12812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:24.885089 12812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:24.904903 12812 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37717
I20260810 01:03:24.904990 12890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37717 every 8 connection(s)
I20260810 01:03:24.906361 12812 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:03:24.910391 12892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:24.913224 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 12812
I20260810 01:03:24.913419 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:03:24.914001 20806 external_mini_cluster.cc:1428] Setting key dc90afd4048a5e8e71acf04d6b36808e
I20260810 01:03:24.922955 12892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2: Bootstrap starting.
I20260810 01:03:24.926604 12892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:24.927728 12895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:24.915918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:57885 (local address 127.0.0.1:37717)
0810 01:03:24.916747 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:24.916760 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:24.916800 (+ 40us) server_negotiation.cc:407] Connection header received
0810 01:03:24.918528 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:24.918551 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:24.918977 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:24.919298 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:24.919922 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:24.921222 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:24.923138 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:24.923818 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:24.923960 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:24.925310 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:24.925318 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:24.925330 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:24.925872 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:24.925950 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:24.925975 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:24.926103 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:24.926514 (+ 411us) server_negotiation.cc:299] Negotiation successful
0810 01:03:24.926704 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":404,"threads_started":1}
I20260810 01:03:24.927721 12892 log.cc:824] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:24.930828 12892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2: No bootstrap required, opened a new log
I20260810 01:03:24.931629 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 01:03:24.939327 12892 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d7c696e2db4c42b77a9eaf7cebc6e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37717 } }
I20260810 01:03:24.939723 12892 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:03:24.939836 12892 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8d7c696e2db4c42b77a9eaf7cebc6e2, State: Initialized, Role: FOLLOWER
I20260810 01:03:24.940459 12892 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d7c696e2db4c42b77a9eaf7cebc6e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37717 } }
I20260810 01:03:24.940758 12892 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:03:24.940920 12892 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:03:24.941116 12892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:24.943142 12895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:24.935876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:33579 (local address 127.0.0.1:37717)
0810 01:03:24.936070 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:24.936076 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:24.936584 (+ 508us) server_negotiation.cc:407] Connection header received
0810 01:03:24.936738 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:24.936743 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:24.936829 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:24.936934 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:24.937427 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:24.938699 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:24.940321 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:24.941047 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:24.941190 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:24.942325 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:24.942333 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:24.942340 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:24.942368 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:24.942460 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:24.942482 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:24.942598 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:24.942747 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 01:03:24.942811 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 01:03:24.943984 12892 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d7c696e2db4c42b77a9eaf7cebc6e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37717 } }
I20260810 01:03:24.944499 12892 leader_election.cc:302] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8d7c696e2db4c42b77a9eaf7cebc6e2; no voters:
I20260810 01:03:24.945072 12892 leader_election.cc:288] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:03:24.945168 12905 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:03:24.945993 12905 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2 [term 1 LEADER]: Becoming Leader. State: Replica: a8d7c696e2db4c42b77a9eaf7cebc6e2, State: Running, Role: LEADER
I20260810 01:03:24.946600 12905 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d7c696e2db4c42b77a9eaf7cebc6e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37717 } }
I20260810 01:03:24.947391 12892 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:24.949715 12907 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8d7c696e2db4c42b77a9eaf7cebc6e2. Latest consensus state: current_term: 1 leader_uuid: "a8d7c696e2db4c42b77a9eaf7cebc6e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d7c696e2db4c42b77a9eaf7cebc6e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37717 } } }
I20260810 01:03:24.950179 12907 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:24.949719 12906 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8d7c696e2db4c42b77a9eaf7cebc6e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d7c696e2db4c42b77a9eaf7cebc6e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37717 } } }
I20260810 01:03:24.952618 12906 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:24.952805 12911 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:03:24.957218 12911 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:03:24.967119 12911 catalog_manager.cc:1347] Generated new cluster ID: 838f937badeb4a129dd0ff37a73b252c
I20260810 01:03:24.967247 12911 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:03:24.988684 12911 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:03:24.990130 12911 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:03:25.000288 12911 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8d7c696e2db4c42b77a9eaf7cebc6e2: Generated new TSK 0
I20260810 01:03:25.001205 12911 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:03:25.007392 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 12812
W20260810 01:03:25.009060 12886 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.44:57885
W20260810 01:03:25.009886 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 12812: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 01:03:25.010246 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 12812
W20260810 01:03:25.033504 12796 connection.cc:535] client connection to 127.0.0.1:37717 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37717): Connection reset by peer (error 104)
2026-08-10T01:03:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2137 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:03:25 dist-test-slave-gp6n krb5kdc[12927](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:03:25 dist-test-slave-gp6n krb5kdc[12927](info): set up 2 sockets
Aug 10 01:03:25 dist-test-slave-gp6n krb5kdc[12927](info): commencing operation
krb5kdc: starting...
W20260810 01:03:26.319469 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.001s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:03:26 dist-test-slave-gp6n krb5kdc[12927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:03:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:03:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 01:03:26.657575 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33699
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:03:27.002804 12943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:27.003089 12943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:27.003197 12943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:27.010288 12943 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:27.010413 12943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:27.010516 12943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:27.010583 12943 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:27.010628 12943 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:27.020437 12943 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33699
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:27.021644 12943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:27.022975 12943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:27.035356 12949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:27.035390 12948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:27.035356 12952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:27.036978 12943 server_base.cc:1034] running on GCE node
I20260810 01:03:27.037688 12943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:27.039355 12943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:27.040611 12943 hybrid_clock.cc:648] HybridClock initialized: now 1786323807040547 us; error 52 us; skew 500 ppm
I20260810 01:03:27.041055 12943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:27 dist-test-slave-gp6n krb5kdc[12927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323807, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:27.044780 12943 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 01:03:27.046453 12943 webserver.cc:460] Webserver started at http://127.0.0.1:40309/ using document root <none> and password file <none>
I20260810 01:03:27.047262 12943 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:27.047412 12943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:27.047755 12943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:27.051155 12943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "b563cfd0a62f41a3b8483f958bfbc690"
format_stamp: "Formatted at 2026-08-10 01:03:27 on dist-test-slave-gp6n"
server_key: "c640ebf19e6354897bf5e451cee98b4d"
server_key_iv: "672160cacb4dd51598e61067f6c90e18"
server_key_version: "encryptionkey@0"
I20260810 01:03:27.052014 12943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "b563cfd0a62f41a3b8483f958bfbc690"
format_stamp: "Formatted at 2026-08-10 01:03:27 on dist-test-slave-gp6n"
server_key: "c640ebf19e6354897bf5e451cee98b4d"
server_key_iv: "672160cacb4dd51598e61067f6c90e18"
server_key_version: "encryptionkey@0"
I20260810 01:03:27.057811 12943 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 01:03:27.061484 12960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:27.062893 12943 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:03:27.063071 12943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "b563cfd0a62f41a3b8483f958bfbc690"
format_stamp: "Formatted at 2026-08-10 01:03:27 on dist-test-slave-gp6n"
server_key: "c640ebf19e6354897bf5e451cee98b4d"
server_key_iv: "672160cacb4dd51598e61067f6c90e18"
server_key_version: "encryptionkey@0"
I20260810 01:03:27.063256 12943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:27.069204 12943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:27.073751 12943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:27.074057 12943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:27.092523 12943 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33699
I20260810 01:03:27.092608 13022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33699 every 8 connection(s)
I20260810 01:03:27.093945 12943 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:03:27.097976 13023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:27.099597 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 12943
I20260810 01:03:27.099792 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:03:27.100240 20806 external_mini_cluster.cc:1428] Setting key ec6ac1dbb4497ea351dfce7be4c3a167
I20260810 01:03:27.109185 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:27.102123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:32899 (local address 127.0.0.1:33699)
0810 01:03:27.103021 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:27.103040 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:27.103097 (+ 57us) server_negotiation.cc:407] Connection header received
0810 01:03:27.105154 (+ 2057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:27.105185 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:27.105658 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:27.106150 (+ 492us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:27.107191 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:27.107200 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:27.107217 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:27.107235 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:27.107299 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:27.107325 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:27.107550 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:27.107691 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 01:03:27.107900 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":337,"threads_started":1}
I20260810 01:03:27.112802 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 01:03:27.113507 13023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690: Bootstrap starting.
I20260810 01:03:27.117741 13023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:27.119177 13023 log.cc:824] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:27.120889 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:27.118527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:37959 (local address 127.0.0.1:33699)
0810 01:03:27.118702 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:27.118708 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:27.119025 (+ 317us) server_negotiation.cc:407] Connection header received
0810 01:03:27.119167 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:27.119175 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:27.119241 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:27.119353 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:27.120275 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:27.120281 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:27.120293 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:27.120307 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:27.120347 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:27.120362 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:27.120474 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:27.120583 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 01:03:27.120642 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 01:03:27.122551 13023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690: No bootstrap required, opened a new log
I20260810 01:03:27.130605 13023 raft_consensus.cc:348] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b563cfd0a62f41a3b8483f958bfbc690" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33699 } }
I20260810 01:03:27.131105 13023 raft_consensus.cc:374] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:03:27.131237 13023 raft_consensus.cc:729] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b563cfd0a62f41a3b8483f958bfbc690, State: Initialized, Role: FOLLOWER
I20260810 01:03:27.131806 13023 consensus_queue.cc:260] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b563cfd0a62f41a3b8483f958bfbc690" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33699 } }
I20260810 01:03:27.132064 13023 raft_consensus.cc:388] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:03:27.132215 13023 raft_consensus.cc:482] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:03:27.132398 13023 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:27.135349 13023 raft_consensus.cc:504] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b563cfd0a62f41a3b8483f958bfbc690" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33699 } }
I20260810 01:03:27.135874 13023 leader_election.cc:302] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b563cfd0a62f41a3b8483f958bfbc690; no voters:
I20260810 01:03:27.136474 13023 leader_election.cc:288] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:03:27.136600 13033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:03:27.137284 13033 raft_consensus.cc:686] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690 [term 1 LEADER]: Becoming Leader. State: Replica: b563cfd0a62f41a3b8483f958bfbc690, State: Running, Role: LEADER
I20260810 01:03:27.137836 13033 consensus_queue.cc:237] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b563cfd0a62f41a3b8483f958bfbc690" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33699 } }
I20260810 01:03:27.138696 13023 sys_catalog.cc:564] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:27.140604 13035 sys_catalog.cc:455] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b563cfd0a62f41a3b8483f958bfbc690. Latest consensus state: current_term: 1 leader_uuid: "b563cfd0a62f41a3b8483f958bfbc690" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b563cfd0a62f41a3b8483f958bfbc690" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33699 } } }
I20260810 01:03:27.141076 13035 sys_catalog.cc:458] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:27.145118 13034 sys_catalog.cc:455] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b563cfd0a62f41a3b8483f958bfbc690" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b563cfd0a62f41a3b8483f958bfbc690" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33699 } } }
I20260810 01:03:27.147431 13034 sys_catalog.cc:458] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:27.148497 13042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:03:27.153637 13042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:03:27.163342 13042 catalog_manager.cc:1347] Generated new cluster ID: fe1dbfb22d674d1ca51c8a8a96933010
I20260810 01:03:27.163501 13042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:03:27.198112 13042 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:03:27.198637 13042 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:03:27.209185 13042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b563cfd0a62f41a3b8483f958bfbc690: Generated new TSK 0
I20260810 01:03:27.210050 13042 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:03:27.216758 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 12943
W20260810 01:03:27.218477 13016 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.44:32899
W20260810 01:03:27.219146 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 12943: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 01:03:27.219304 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 12943
2026-08-10T01:03:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2205 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:03:27 dist-test-slave-gp6n krb5kdc[13056](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:03:27 dist-test-slave-gp6n krb5kdc[13056](info): set up 2 sockets
Aug 10 01:03:27 dist-test-slave-gp6n krb5kdc[13056](info): commencing operation
krb5kdc: starting...
W20260810 01:03:28.424911 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.004s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:03:28 dist-test-slave-gp6n krb5kdc[13056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:03:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:03:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 01:03:28.769378 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:33991
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:37653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:33991
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:03:29.119413 13074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:29.119766 13074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:29.120041 13074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:29.127388 13074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:29.127573 13074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:29.127676 13074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:29.127789 13074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:29.127868 13074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:29.137598 13074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:37653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:33991
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:33991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:29.138929 13074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:29.140268 13074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:29.152135 13080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:29.152170 13084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:29.152207 13081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:29.153043 13074 server_base.cc:1034] running on GCE node
I20260810 01:03:29.153829 13074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:29.155336 13074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:29.156577 13074 hybrid_clock.cc:648] HybridClock initialized: now 1786323809156530 us; error 42 us; skew 500 ppm
I20260810 01:03:29.156924 13074 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:29 dist-test-slave-gp6n krb5kdc[13056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323809, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:29.160214 13074 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 01:03:29.161728 13074 webserver.cc:460] Webserver started at http://127.20.81.190:37393/ using document root <none> and password file <none>
I20260810 01:03:29.162478 13074 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:29.162632 13074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:29.162946 13074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:29.165876 13074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "457e8aa999424eb499079aff5af69976"
format_stamp: "Formatted at 2026-08-10 01:03:29 on dist-test-slave-gp6n"
server_key: "de6095f11b054f280d657db2ece604e7"
server_key_iv: "674eec598b096288eee5efdcdb07c45b"
server_key_version: "encryptionkey@0"
I20260810 01:03:29.166705 13074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "457e8aa999424eb499079aff5af69976"
format_stamp: "Formatted at 2026-08-10 01:03:29 on dist-test-slave-gp6n"
server_key: "de6095f11b054f280d657db2ece604e7"
server_key_iv: "674eec598b096288eee5efdcdb07c45b"
server_key_version: "encryptionkey@0"
I20260810 01:03:29.171844 13074 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 01:03:29.175398 13092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:29.176685 13074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 01:03:29.176872 13074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "457e8aa999424eb499079aff5af69976"
format_stamp: "Formatted at 2026-08-10 01:03:29 on dist-test-slave-gp6n"
server_key: "de6095f11b054f280d657db2ece604e7"
server_key_iv: "674eec598b096288eee5efdcdb07c45b"
server_key_version: "encryptionkey@0"
I20260810 01:03:29.177068 13074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:29.191071 13074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:29.195283 13074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:29.195565 13074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:29.214705 13074 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:33991
I20260810 01:03:29.214738 13150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:33991 every 8 connection(s)
I20260810 01:03:29.216176 13074 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:03:29.220093 13151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:29.224917 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 13074
I20260810 01:03:29.225108 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:03:29.225548 20806 external_mini_cluster.cc:1428] Setting key f44abfdb312f6502274f5798c6cc2ecd
I20260810 01:03:29.231247 13151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976: Bootstrap starting.
I20260810 01:03:29.234866 13151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:29.236091 13151 log.cc:824] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:29.239110 13151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976: No bootstrap required, opened a new log
Aug 10 01:03:29 dist-test-slave-gp6n krb5kdc[13056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:03:29.250038 13151 raft_consensus.cc:348] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "457e8aa999424eb499079aff5af69976" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33991 } }
I20260810 01:03:29.250528 13151 raft_consensus.cc:374] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:03:29.250640 13151 raft_consensus.cc:729] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 457e8aa999424eb499079aff5af69976, State: Initialized, Role: FOLLOWER
I20260810 01:03:29.250878 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:29.227351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41215 (local address 127.20.81.190:33991)
0810 01:03:29.228166 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:29.228177 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:29.228222 (+ 45us) server_negotiation.cc:407] Connection header received
0810 01:03:29.229958 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:29.229980 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:29.230340 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:29.230730 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:29.232561 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:29.233873 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:29.235743 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:29.236488 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:29.236719 (+ 231us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:29.241476 (+ 4757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:29.241486 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:29.241509 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:29.241565 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:29.245782 (+ 4217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:29.246770 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:29.246784 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:29.246800 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:29.247007 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:29.247500 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:29.247507 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:29.247515 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:29.248309 (+ 794us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:29.248659 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:29.249101 (+ 442us) server_negotiation.cc:299] Negotiation successful
0810 01:03:29.249457 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":381,"threads_started":1}
I20260810 01:03:29.251349 13151 consensus_queue.cc:260] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "457e8aa999424eb499079aff5af69976" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33991 } }
I20260810 01:03:29.251601 13151 raft_consensus.cc:388] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:03:29.251763 13151 raft_consensus.cc:482] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:03:29.251956 13151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:29.255198 13151 raft_consensus.cc:504] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "457e8aa999424eb499079aff5af69976" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33991 } }
I20260810 01:03:29.255338 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 01:03:29.255787 13151 leader_election.cc:302] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 457e8aa999424eb499079aff5af69976; no voters:
I20260810 01:03:29.256418 13151 leader_election.cc:288] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:03:29.256542 13156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:03:29.257850 13156 raft_consensus.cc:686] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976 [term 1 LEADER]: Becoming Leader. State: Replica: 457e8aa999424eb499079aff5af69976, State: Running, Role: LEADER
I20260810 01:03:29.258463 13156 consensus_queue.cc:237] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "457e8aa999424eb499079aff5af69976" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33991 } }
I20260810 01:03:29.258700 13151 sys_catalog.cc:564] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:29.261168 13158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 457e8aa999424eb499079aff5af69976. Latest consensus state: current_term: 1 leader_uuid: "457e8aa999424eb499079aff5af69976" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "457e8aa999424eb499079aff5af69976" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33991 } } }
I20260810 01:03:29.261550 13158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:29.262115 13157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "457e8aa999424eb499079aff5af69976" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "457e8aa999424eb499079aff5af69976" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33991 } } }
I20260810 01:03:29.262480 13157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:29.271253 13165 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:03:29.276304 13165 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:03:29.302156 13165 catalog_manager.cc:1347] Generated new cluster ID: fc20caf10bca465ebd1978f22178f93c
I20260810 01:03:29.302279 13165 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:03:29.305534 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:29.278650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43233 (local address 127.20.81.190:33991)
0810 01:03:29.278903 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:29.278912 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:29.281581 (+ 2669us) server_negotiation.cc:407] Connection header received
0810 01:03:29.281928 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:29.281938 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:29.282040 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:29.282181 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:29.284835 (+ 2654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:29.286352 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:29.288763 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:29.289722 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:29.289939 (+ 217us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:29.294716 (+ 4777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:29.294724 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:29.294728 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:29.294766 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:29.299537 (+ 4771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:29.300855 (+ 1318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:29.300867 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:29.300874 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:29.301062 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:29.301771 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:29.301779 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:29.301784 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:29.302449 (+ 665us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:29.302932 (+ 483us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:29.304734 (+ 1802us) server_negotiation.cc:299] Negotiation successful
0810 01:03:29.304995 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 01:03:29.320175 13165 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:03:29.321619 13165 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:03:29.336108 13165 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 457e8aa999424eb499079aff5af69976: Generated new TSK 0
I20260810 01:03:29.337294 13165 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:03:29.349567 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 13074
I20260810 01:03:29.351073 13146 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:03:29.446240 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 13074
W20260810 01:03:29.468520 13054 connection.cc:535] client connection to 127.20.81.190:33991 recv error: Network error: failed to read from TLS socket (remote: 127.20.81.190:33991): Connection reset by peer (error 104)
2026-08-10T01:03:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2229 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:03:29 dist-test-slave-gp6n krb5kdc[13190](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:03:29 dist-test-slave-gp6n krb5kdc[13190](info): set up 2 sockets
Aug 10 01:03:29 dist-test-slave-gp6n krb5kdc[13190](info): commencing operation
krb5kdc: starting...
W20260810 01:03:30.668148 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:03:30 dist-test-slave-gp6n krb5kdc[13190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:03:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:03:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 01:03:31.026180 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:33231
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:42003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:33231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:03:31.377219 13207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:31.377527 13207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:31.377712 13207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:31.385114 13207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:31.385251 13207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:31.385356 13207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:31.385458 13207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:31.385519 13207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:31.395521 13207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:42003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:33231
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:33231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:31.396873 13207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:31.398272 13207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:31.410771 13212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:31.410812 13213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:31.410849 13215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:31.411381 13207 server_base.cc:1034] running on GCE node
I20260810 01:03:31.412274 13207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:31.413753 13207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:31.414981 13207 hybrid_clock.cc:648] HybridClock initialized: now 1786323811414920 us; error 58 us; skew 500 ppm
I20260810 01:03:31.415328 13207 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:31 dist-test-slave-gp6n krb5kdc[13190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323811, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:31.423400 13207 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 01:03:31.424901 13207 webserver.cc:460] Webserver started at http://127.20.81.190:43037/ using document root <none> and password file <none>
I20260810 01:03:31.425551 13207 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:31.425655 13207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:31.425915 13207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:31.429003 13207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "af853b92cae842baaeade7b17ec948e2"
format_stamp: "Formatted at 2026-08-10 01:03:31 on dist-test-slave-gp6n"
server_key: "ab88b7d4ccf85fcee590d16ede6555b9"
server_key_iv: "b927d302a1f1ed912d067f5bccd71f1b"
server_key_version: "encryptionkey@0"
I20260810 01:03:31.429716 13207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "af853b92cae842baaeade7b17ec948e2"
format_stamp: "Formatted at 2026-08-10 01:03:31 on dist-test-slave-gp6n"
server_key: "ab88b7d4ccf85fcee590d16ede6555b9"
server_key_iv: "b927d302a1f1ed912d067f5bccd71f1b"
server_key_version: "encryptionkey@0"
I20260810 01:03:31.434860 13207 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260810 01:03:31.438206 13222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:31.439845 13207 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:03:31.439998 13207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "af853b92cae842baaeade7b17ec948e2"
format_stamp: "Formatted at 2026-08-10 01:03:31 on dist-test-slave-gp6n"
server_key: "ab88b7d4ccf85fcee590d16ede6555b9"
server_key_iv: "b927d302a1f1ed912d067f5bccd71f1b"
server_key_version: "encryptionkey@0"
I20260810 01:03:31.440156 13207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:31.454483 13207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:31.458606 13207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:31.458856 13207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:31.477741 13207 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:33231
I20260810 01:03:31.477813 13282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:33231 every 8 connection(s)
I20260810 01:03:31.479161 13207 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:03:31.479928 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 13207
I20260810 01:03:31.480165 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:03:31.480719 20806 external_mini_cluster.cc:1428] Setting key 81a29dfee6d275e4cfbafb44f44f7f93
I20260810 01:03:31.484570 13283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:31.497737 13286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:31.482693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44089 (local address 127.20.81.190:33231)
0810 01:03:31.483595 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:31.483613 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:31.483670 (+ 57us) server_negotiation.cc:407] Connection header received
0810 01:03:31.486378 (+ 2708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:31.486447 (+ 69us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:31.486997 (+ 550us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:31.487497 (+ 500us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:31.488091 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:31.490442 (+ 2351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:31.491858 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:31.493011 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:31.493198 (+ 187us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:31.494596 (+ 1398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:31.494609 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:31.494628 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:31.495430 (+ 802us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:31.495542 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:31.495581 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:31.495783 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:31.496042 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 01:03:31.496244 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":344,"threads_started":1}
I20260810 01:03:31.500206 13283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2: Bootstrap starting.
I20260810 01:03:31.501839 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 01:03:31.503773 13283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:31.504889 13283 log.cc:824] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:31.508365 13283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2: No bootstrap required, opened a new log
I20260810 01:03:31.514637 13286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:31.506948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38563 (local address 127.20.81.190:33231)
0810 01:03:31.507139 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:31.507145 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:31.507555 (+ 410us) server_negotiation.cc:407] Connection header received
0810 01:03:31.507714 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:31.507720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:31.507809 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:31.507934 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:31.508472 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:31.509856 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:31.511407 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:31.512240 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:31.512391 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:31.513655 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:31.513662 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:31.513668 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:31.513690 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:31.513744 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:31.513775 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:31.513917 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:31.514155 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 01:03:31.514223 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 01:03:31.518688 13283 raft_consensus.cc:348] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af853b92cae842baaeade7b17ec948e2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33231 } }
I20260810 01:03:31.519078 13283 raft_consensus.cc:374] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:03:31.519157 13283 raft_consensus.cc:729] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af853b92cae842baaeade7b17ec948e2, State: Initialized, Role: FOLLOWER
I20260810 01:03:31.519766 13283 consensus_queue.cc:260] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af853b92cae842baaeade7b17ec948e2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33231 } }
I20260810 01:03:31.520128 13283 raft_consensus.cc:388] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:03:31.520318 13283 raft_consensus.cc:482] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:03:31.520543 13283 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:31.523128 13283 raft_consensus.cc:504] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af853b92cae842baaeade7b17ec948e2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33231 } }
I20260810 01:03:31.523605 13283 leader_election.cc:302] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af853b92cae842baaeade7b17ec948e2; no voters:
I20260810 01:03:31.524202 13283 leader_election.cc:288] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:03:31.524363 13293 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:03:31.525141 13293 raft_consensus.cc:686] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2 [term 1 LEADER]: Becoming Leader. State: Replica: af853b92cae842baaeade7b17ec948e2, State: Running, Role: LEADER
I20260810 01:03:31.525679 13293 consensus_queue.cc:237] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af853b92cae842baaeade7b17ec948e2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33231 } }
I20260810 01:03:31.526692 13283 sys_catalog.cc:564] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:31.528622 13295 sys_catalog.cc:455] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af853b92cae842baaeade7b17ec948e2. Latest consensus state: current_term: 1 leader_uuid: "af853b92cae842baaeade7b17ec948e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af853b92cae842baaeade7b17ec948e2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33231 } } }
I20260810 01:03:31.529186 13295 sys_catalog.cc:458] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:31.529093 13294 sys_catalog.cc:455] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af853b92cae842baaeade7b17ec948e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af853b92cae842baaeade7b17ec948e2" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 33231 } } }
I20260810 01:03:31.529458 13294 sys_catalog.cc:458] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:31.531422 13300 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:03:31.537001 13300 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:03:31.547611 13300 catalog_manager.cc:1347] Generated new cluster ID: 49263023b0184bef9cd5a6fde8c92c3b
I20260810 01:03:31.547765 13300 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:03:31.560359 13300 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:03:31.561894 13300 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:03:31.573678 13300 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af853b92cae842baaeade7b17ec948e2: Generated new TSK 0
I20260810 01:03:31.574717 13300 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:03:31.582890 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 13207
W20260810 01:03:31.584154 13277 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44089
W20260810 01:03:31.584991 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 13207: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 01:03:31.585186 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 13207
W20260810 01:03:31.609833 13188 connection.cc:535] client connection to 127.20.81.190:33231 recv error: Network error: failed to read from TLS socket (remote: 127.20.81.190:33231): Connection reset by peer (error 104)
2026-08-10T01:03:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2141 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:03:31 dist-test-slave-gp6n krb5kdc[13317](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:03:31 dist-test-slave-gp6n krb5kdc[13317](info): set up 2 sockets
Aug 10 01:03:31 dist-test-slave-gp6n krb5kdc[13317](info): commencing operation
krb5kdc: starting...
W20260810 01:03:32.852836 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.002s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:03:33 dist-test-slave-gp6n krb5kdc[13317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:03:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:03:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 01:03:33.194293 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:39839
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:39967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:39839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:03:33.542695 13332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:33.543040 13332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:33.543211 13332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:33.550338 13332 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:33.550508 13332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:33.550611 13332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:33.550719 13332 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:33.550793 13332 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:33.560518 13332 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:39967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:39839
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:39839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:33.561882 13332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:33.563301 13332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:33.575425 13339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:33.575562 13340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:33.575585 13342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:33.576363 13332 server_base.cc:1034] running on GCE node
I20260810 01:03:33.577054 13332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:33.578490 13332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:33.579712 13332 hybrid_clock.cc:648] HybridClock initialized: now 1786323813579673 us; error 29 us; skew 500 ppm
I20260810 01:03:33.580068 13332 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:33 dist-test-slave-gp6n krb5kdc[13317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323813, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:33.583189 13332 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 01:03:33.584728 13332 webserver.cc:460] Webserver started at http://127.20.81.190:45139/ using document root <none> and password file <none>
I20260810 01:03:33.585400 13332 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:33.585541 13332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:33.585853 13332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:33.588892 13332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "b4d10a16ad214f27ad6f182f0393c2e9"
format_stamp: "Formatted at 2026-08-10 01:03:33 on dist-test-slave-gp6n"
server_key: "7a1ca3d7f5015361a99a6b3a0dd637e9"
server_key_iv: "276ad9c62ba8aac13c40dd9cb4cdb513"
server_key_version: "encryptionkey@0"
I20260810 01:03:33.589672 13332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "b4d10a16ad214f27ad6f182f0393c2e9"
format_stamp: "Formatted at 2026-08-10 01:03:33 on dist-test-slave-gp6n"
server_key: "7a1ca3d7f5015361a99a6b3a0dd637e9"
server_key_iv: "276ad9c62ba8aac13c40dd9cb4cdb513"
server_key_version: "encryptionkey@0"
I20260810 01:03:33.594766 13332 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260810 01:03:33.598155 13352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:33.599541 13332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:03:33.599728 13332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "b4d10a16ad214f27ad6f182f0393c2e9"
format_stamp: "Formatted at 2026-08-10 01:03:33 on dist-test-slave-gp6n"
server_key: "7a1ca3d7f5015361a99a6b3a0dd637e9"
server_key_iv: "276ad9c62ba8aac13c40dd9cb4cdb513"
server_key_version: "encryptionkey@0"
I20260810 01:03:33.599936 13332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:33.605719 13332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:33.609716 13332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:33.610014 13332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:33.629433 13332 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:39839
I20260810 01:03:33.629407 13414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:39839 every 8 connection(s)
I20260810 01:03:33.630918 13332 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:03:33.634769 13416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:33.636273 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 13332
I20260810 01:03:33.636476 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:03:33.637065 20806 external_mini_cluster.cc:1428] Setting key 503689fddf2b794b83b0411027fc1dc3
I20260810 01:03:33.644778 13420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:33.638924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37797 (local address 127.20.81.190:39839)
0810 01:03:33.639766 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:33.639777 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:33.639825 (+ 48us) server_negotiation.cc:407] Connection header received
0810 01:03:33.641427 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:33.641452 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:33.641799 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:33.642142 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:33.643232 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:33.643239 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:33.643252 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:33.643267 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:33.643331 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:33.643360 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:33.643534 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:33.643661 (+ 127us) server_negotiation.cc:299] Negotiation successful
0810 01:03:33.643873 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":396,"threads_started":1}
I20260810 01:03:33.648945 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 01:03:33.648972 13416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9: Bootstrap starting.
I20260810 01:03:33.652884 13416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:33.654249 13416 log.cc:824] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:33.657321 13420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:33.654466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48987 (local address 127.20.81.190:39839)
0810 01:03:33.654647 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:33.654654 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:33.655277 (+ 623us) server_negotiation.cc:407] Connection header received
0810 01:03:33.655438 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:33.655446 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:33.655513 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:33.655632 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:33.656670 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:33.656673 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:33.656676 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:33.656691 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:33.656725 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:33.656739 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:33.656828 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:33.656999 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 01:03:33.657042 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 01:03:33.658048 13416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9: No bootstrap required, opened a new log
I20260810 01:03:33.667097 13416 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d10a16ad214f27ad6f182f0393c2e9" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39839 } }
I20260810 01:03:33.667452 13416 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:03:33.667533 13416 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4d10a16ad214f27ad6f182f0393c2e9, State: Initialized, Role: FOLLOWER
I20260810 01:03:33.668068 13416 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d10a16ad214f27ad6f182f0393c2e9" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39839 } }
I20260810 01:03:33.668299 13416 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:03:33.668421 13416 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:03:33.668565 13416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:33.671020 13416 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d10a16ad214f27ad6f182f0393c2e9" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39839 } }
I20260810 01:03:33.671586 13416 leader_election.cc:302] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4d10a16ad214f27ad6f182f0393c2e9; no voters:
I20260810 01:03:33.672432 13416 leader_election.cc:288] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:03:33.672592 13427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:03:33.673420 13427 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9 [term 1 LEADER]: Becoming Leader. State: Replica: b4d10a16ad214f27ad6f182f0393c2e9, State: Running, Role: LEADER
I20260810 01:03:33.674000 13427 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d10a16ad214f27ad6f182f0393c2e9" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39839 } }
I20260810 01:03:33.674933 13416 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:33.677134 13428 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4d10a16ad214f27ad6f182f0393c2e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d10a16ad214f27ad6f182f0393c2e9" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39839 } } }
I20260810 01:03:33.677685 13428 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:33.678390 13429 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4d10a16ad214f27ad6f182f0393c2e9. Latest consensus state: current_term: 1 leader_uuid: "b4d10a16ad214f27ad6f182f0393c2e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d10a16ad214f27ad6f182f0393c2e9" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 39839 } } }
I20260810 01:03:33.684898 13429 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:33.687528 13437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:03:33.694695 13437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:03:33.705756 13437 catalog_manager.cc:1347] Generated new cluster ID: 2a2f29014021493f9dafc73222eb9d37
I20260810 01:03:33.705879 13437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:03:33.763342 13437 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:03:33.763856 13437 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:03:33.777917 13437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4d10a16ad214f27ad6f182f0393c2e9: Generated new TSK 0
I20260810 01:03:33.779165 13437 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:03:33.801194 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 13332
W20260810 01:03:33.802701 13410 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37797
W20260810 01:03:33.803414 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 13332: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 01:03:33.803601 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 13332
2026-08-10T01:03:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2218 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:03:33 dist-test-slave-gp6n krb5kdc[13450](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:03:33 dist-test-slave-gp6n krb5kdc[13450](info): set up 2 sockets
Aug 10 01:03:33 dist-test-slave-gp6n krb5kdc[13450](info): commencing operation
krb5kdc: starting...
W20260810 01:03:35.028367 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:03:35 dist-test-slave-gp6n krb5kdc[13450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:03:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:03:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 01:03:35.418381 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:36731
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:45089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:36731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:03:35.788071 13465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:35.788349 13465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:35.788553 13465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:35.795961 13465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:35.796082 13465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:35.796196 13465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:35.796283 13465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:35.796360 13465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:35.806664 13465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:45089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:36731
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:36731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:35.807996 13465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:35.809396 13465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:35.821981 13472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:35.822059 13475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:35.821947 13473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:35.823312 13465 server_base.cc:1034] running on GCE node
I20260810 01:03:35.824097 13465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:35.825539 13465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:35.826748 13465 hybrid_clock.cc:648] HybridClock initialized: now 1786323815826699 us; error 45 us; skew 500 ppm
I20260810 01:03:35.827140 13465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:35 dist-test-slave-gp6n krb5kdc[13450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323815, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:35.830525 13465 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 01:03:35.832234 13465 webserver.cc:460] Webserver started at http://127.20.81.190:43061/ using document root <none> and password file <none>
I20260810 01:03:35.832970 13465 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:35.833125 13465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:35.833462 13465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:35.836655 13465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "3f18aee6db6a4cea95c927b335662bdc"
format_stamp: "Formatted at 2026-08-10 01:03:35 on dist-test-slave-gp6n"
server_key: "4fdd2fe3cb722b5cf6a8df9dfa27b2ce"
server_key_iv: "e4207250d96511b8993e98454cef6790"
server_key_version: "encryptionkey@0"
I20260810 01:03:35.837486 13465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "3f18aee6db6a4cea95c927b335662bdc"
format_stamp: "Formatted at 2026-08-10 01:03:35 on dist-test-slave-gp6n"
server_key: "4fdd2fe3cb722b5cf6a8df9dfa27b2ce"
server_key_iv: "e4207250d96511b8993e98454cef6790"
server_key_version: "encryptionkey@0"
I20260810 01:03:35.843173 13465 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 01:03:35.846823 13483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:35.848253 13465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 01:03:35.848449 13465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "3f18aee6db6a4cea95c927b335662bdc"
format_stamp: "Formatted at 2026-08-10 01:03:35 on dist-test-slave-gp6n"
server_key: "4fdd2fe3cb722b5cf6a8df9dfa27b2ce"
server_key_iv: "e4207250d96511b8993e98454cef6790"
server_key_version: "encryptionkey@0"
I20260810 01:03:35.848666 13465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:35.861286 13465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:35.865581 13465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:35.865881 13465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:35.885375 13465 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:36731
I20260810 01:03:35.885416 13543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:36731 every 8 connection(s)
I20260810 01:03:35.886905 13465 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:03:35.890681 13544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:35.892933 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 13465
I20260810 01:03:35.893128 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:03:35.893678 20806 external_mini_cluster.cc:1428] Setting key 65f705c9e1580176dc82f5b7d00d98e4
I20260810 01:03:35.903528 13544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc: Bootstrap starting.
I20260810 01:03:35.907644 13544 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc: Neither blocks nor log segments found. Creating new log.
Aug 10 01:03:35 dist-test-slave-gp6n krb5kdc[13450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:03:35.908895 13544 log.cc:824] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:35.912349 13544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc: No bootstrap required, opened a new log
I20260810 01:03:35.917667 13547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:35.895621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55677 (local address 127.20.81.190:36731)
0810 01:03:35.896466 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:35.896479 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:35.896535 (+ 56us) server_negotiation.cc:407] Connection header received
0810 01:03:35.898213 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:35.898236 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:35.898709 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:35.899037 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:35.900958 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:35.902280 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:35.904098 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:35.904768 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:35.904900 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:35.909907 (+ 5007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:35.909921 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:35.909952 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:35.910020 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:35.913324 (+ 3304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:35.914174 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:35.914180 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:35.914186 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:35.914375 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:35.914892 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:35.914896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:35.914898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:35.915455 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:35.915666 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:35.916158 (+ 492us) server_negotiation.cc:299] Negotiation successful
0810 01:03:35.916588 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":408,"threads_started":1}
I20260810 01:03:35.921583 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 01:03:35.921888 13544 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f18aee6db6a4cea95c927b335662bdc" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 36731 } }
I20260810 01:03:35.922254 13544 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:03:35.922360 13544 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f18aee6db6a4cea95c927b335662bdc, State: Initialized, Role: FOLLOWER
I20260810 01:03:35.923024 13544 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f18aee6db6a4cea95c927b335662bdc" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 36731 } }
I20260810 01:03:35.923270 13544 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:03:35.923420 13544 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:03:35.923614 13544 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:35.925791 13544 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f18aee6db6a4cea95c927b335662bdc" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 36731 } }
I20260810 01:03:35.926380 13544 leader_election.cc:302] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f18aee6db6a4cea95c927b335662bdc; no voters:
I20260810 01:03:35.927181 13544 leader_election.cc:288] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:03:35.927296 13555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:03:35.927985 13555 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc [term 1 LEADER]: Becoming Leader. State: Replica: 3f18aee6db6a4cea95c927b335662bdc, State: Running, Role: LEADER
I20260810 01:03:35.928607 13555 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f18aee6db6a4cea95c927b335662bdc" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 36731 } }
I20260810 01:03:35.931919 13544 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:35.932240 13557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f18aee6db6a4cea95c927b335662bdc. Latest consensus state: current_term: 1 leader_uuid: "3f18aee6db6a4cea95c927b335662bdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f18aee6db6a4cea95c927b335662bdc" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 36731 } } }
I20260810 01:03:35.932619 13557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:35.937464 13559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f18aee6db6a4cea95c927b335662bdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f18aee6db6a4cea95c927b335662bdc" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 36731 } } }
I20260810 01:03:35.938472 13559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:35.944232 13561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:03:35.947130 13547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:35.926258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38015 (local address 127.20.81.190:36731)
0810 01:03:35.926531 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:35.926537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:35.926949 (+ 412us) server_negotiation.cc:407] Connection header received
0810 01:03:35.927172 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:35.927178 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:35.927263 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:35.927397 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:35.929734 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:35.931061 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:35.932913 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:35.933781 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:35.933943 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:35.938181 (+ 4238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:35.938191 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:35.938199 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:35.938251 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:35.943106 (+ 4855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:35.944227 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:35.944235 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:35.944241 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:35.944439 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:35.944909 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:35.944917 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:35.944922 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:35.945523 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:35.945848 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:35.946238 (+ 390us) server_negotiation.cc:299] Negotiation successful
0810 01:03:35.946568 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 01:03:35.951719 13561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:03:35.960922 13561 catalog_manager.cc:1347] Generated new cluster ID: 9ce7e4ec5969437da1a2316ce74bd42f
I20260810 01:03:35.961097 13561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:03:35.973991 13561 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:03:35.975425 13561 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:03:35.981798 13561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f18aee6db6a4cea95c927b335662bdc: Generated new TSK 0
I20260810 01:03:35.982851 13561 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:03:35.993875 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 13465
I20260810 01:03:35.995401 13539 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:03:36.086184 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 13465
2026-08-10T01:03:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2278 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:03:36 dist-test-slave-gp6n krb5kdc[13582](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:03:36 dist-test-slave-gp6n krb5kdc[13582](info): set up 2 sockets
Aug 10 01:03:36 dist-test-slave-gp6n krb5kdc[13582](info): commencing operation
krb5kdc: starting...
W20260810 01:03:37.325197 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.004s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:03:37 dist-test-slave-gp6n krb5kdc[13582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:03:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:03:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 01:03:37.688076 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:46227
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:37119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:46227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:03:38.077476 13598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:38.077872 13598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:38.078054 13598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:38.085495 13598 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:38.085659 13598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:38.085777 13598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:38.085893 13598 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:38.085986 13598 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:38.096428 13598 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:37119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:46227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:46227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:38.097867 13598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:38.099296 13598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:38.112126 13604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:38.112188 13607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:38.112226 13605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:38.113763 13598 server_base.cc:1034] running on GCE node
I20260810 01:03:38.114670 13598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:38.116199 13598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:38.117455 13598 hybrid_clock.cc:648] HybridClock initialized: now 1786323818117395 us; error 44 us; skew 500 ppm
I20260810 01:03:38.117851 13598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:38 dist-test-slave-gp6n krb5kdc[13582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323818, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:38.126291 13598 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 01:03:38.128075 13598 webserver.cc:460] Webserver started at http://127.20.81.190:33749/ using document root <none> and password file <none>
I20260810 01:03:38.128854 13598 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:38.129011 13598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:38.129383 13598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:38.132892 13598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "dafd9228bf7540e39d0f8eac6f375d3f"
format_stamp: "Formatted at 2026-08-10 01:03:38 on dist-test-slave-gp6n"
server_key: "9807a4c0fca4b48d2e59ae6b4f620c9d"
server_key_iv: "feb4c16a027f70103453bdffa8ba4b7e"
server_key_version: "encryptionkey@0"
I20260810 01:03:38.133769 13598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "dafd9228bf7540e39d0f8eac6f375d3f"
format_stamp: "Formatted at 2026-08-10 01:03:38 on dist-test-slave-gp6n"
server_key: "9807a4c0fca4b48d2e59ae6b4f620c9d"
server_key_iv: "feb4c16a027f70103453bdffa8ba4b7e"
server_key_version: "encryptionkey@0"
I20260810 01:03:38.139247 13598 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260810 01:03:38.142930 13615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:38.144412 13598 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 01:03:38.144644 13598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "dafd9228bf7540e39d0f8eac6f375d3f"
format_stamp: "Formatted at 2026-08-10 01:03:38 on dist-test-slave-gp6n"
server_key: "9807a4c0fca4b48d2e59ae6b4f620c9d"
server_key_iv: "feb4c16a027f70103453bdffa8ba4b7e"
server_key_version: "encryptionkey@0"
I20260810 01:03:38.144865 13598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:38.175526 13598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:38.180526 13598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:38.180855 13598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:38.201586 13598 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:46227
I20260810 01:03:38.201622 13671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:46227 every 8 connection(s)
I20260810 01:03:38.203136 13598 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:03:38.204857 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 13598
I20260810 01:03:38.205049 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:03:38.205631 20806 external_mini_cluster.cc:1428] Setting key b22d8eead68e9ea704738441654826b7
I20260810 01:03:38.207911 13678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:38.221243 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:38.207578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53843 (local address 127.20.81.190:46227)
0810 01:03:38.208435 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:38.208448 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:38.208490 (+ 42us) server_negotiation.cc:407] Connection header received
0810 01:03:38.210249 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:38.210270 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:38.210769 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:38.211128 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:38.211781 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:38.213304 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:38.215157 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:38.216001 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:38.216263 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:38.217738 (+ 1475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:38.217752 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:38.217773 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:38.218728 (+ 955us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:38.218858 (+ 130us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:38.218898 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:38.219069 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:38.219291 (+ 222us) server_negotiation.cc:299] Negotiation successful
0810 01:03:38.219634 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":405,"threads_started":1}
I20260810 01:03:38.223059 13678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f: Bootstrap starting.
I20260810 01:03:38.227182 13678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:38.227216 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 01:03:38.228312 13678 log.cc:824] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:38.231774 13678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f: No bootstrap required, opened a new log
I20260810 01:03:38.239904 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:38.232541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47325 (local address 127.20.81.190:46227)
0810 01:03:38.232771 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:38.232778 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:38.233129 (+ 351us) server_negotiation.cc:407] Connection header received
0810 01:03:38.233388 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:38.233394 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:38.233480 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:38.233615 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:38.234342 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:38.235648 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:38.237121 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:38.237647 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:38.237794 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:38.239095 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:38.239102 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:38.239107 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:38.239135 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:38.239176 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:38.239193 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:38.239300 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:38.239463 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 01:03:38.239561 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 01:03:38.239833 13678 raft_consensus.cc:348] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafd9228bf7540e39d0f8eac6f375d3f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 46227 } }
I20260810 01:03:38.240356 13678 raft_consensus.cc:374] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:03:38.240478 13678 raft_consensus.cc:729] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dafd9228bf7540e39d0f8eac6f375d3f, State: Initialized, Role: FOLLOWER
I20260810 01:03:38.241168 13678 consensus_queue.cc:260] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafd9228bf7540e39d0f8eac6f375d3f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 46227 } }
I20260810 01:03:38.241447 13678 raft_consensus.cc:388] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:03:38.241598 13678 raft_consensus.cc:482] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:03:38.241755 13678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:38.244585 13678 raft_consensus.cc:504] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafd9228bf7540e39d0f8eac6f375d3f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 46227 } }
I20260810 01:03:38.245031 13678 leader_election.cc:302] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dafd9228bf7540e39d0f8eac6f375d3f; no voters:
I20260810 01:03:38.245595 13678 leader_election.cc:288] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:03:38.245765 13690 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:03:38.246599 13690 raft_consensus.cc:686] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f [term 1 LEADER]: Becoming Leader. State: Replica: dafd9228bf7540e39d0f8eac6f375d3f, State: Running, Role: LEADER
I20260810 01:03:38.247213 13690 consensus_queue.cc:237] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafd9228bf7540e39d0f8eac6f375d3f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 46227 } }
I20260810 01:03:38.247800 13678 sys_catalog.cc:564] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:38.252287 13692 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader dafd9228bf7540e39d0f8eac6f375d3f. Latest consensus state: current_term: 1 leader_uuid: "dafd9228bf7540e39d0f8eac6f375d3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafd9228bf7540e39d0f8eac6f375d3f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 46227 } } }
I20260810 01:03:38.252353 13691 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dafd9228bf7540e39d0f8eac6f375d3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafd9228bf7540e39d0f8eac6f375d3f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 46227 } } }
I20260810 01:03:38.252810 13692 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:38.252938 13691 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:38.254300 13695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:03:38.259560 13695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:03:38.272955 13695 catalog_manager.cc:1347] Generated new cluster ID: 23ec9b7cbcf645c39d8c40e012d8dcee
I20260810 01:03:38.273113 13695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:03:38.294684 13695 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:03:38.296727 13695 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:03:38.307780 13695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dafd9228bf7540e39d0f8eac6f375d3f: Generated new TSK 0
I20260810 01:03:38.308683 13695 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:03:38.321504 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 13598
W20260810 01:03:38.322820 13667 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53843
W20260810 01:03:38.323580 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 13598: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 01:03:38.323758 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 13598
2026-08-10T01:03:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2238 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:03:38 dist-test-slave-gp6n krb5kdc[13716](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:03:38 dist-test-slave-gp6n krb5kdc[13716](info): set up 2 sockets
Aug 10 01:03:38 dist-test-slave-gp6n krb5kdc[13716](info): commencing operation
krb5kdc: starting...
W20260810 01:03:39.611439 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:03:39 dist-test-slave-gp6n krb5kdc[13716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:03:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:03:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 01:03:39.967212 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:35701
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:36595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:35701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:03:40.323529 13730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:40.323840 13730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:40.324021 13730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:40.331470 13730 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:40.331585 13730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:40.331702 13730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:40.331820 13730 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:40.331898 13730 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:40.341953 13730 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:36595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:35701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:35701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:40.343406 13730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:40.344765 13730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:40.357576 13735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:40.357532 13738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:40.357625 13736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:40.358532 13730 server_base.cc:1034] running on GCE node
I20260810 01:03:40.359380 13730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:40.360895 13730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:40.362111 13730 hybrid_clock.cc:648] HybridClock initialized: now 1786323820362067 us; error 34 us; skew 500 ppm
I20260810 01:03:40.362545 13730 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:40 dist-test-slave-gp6n krb5kdc[13716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323820, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:40.365691 13730 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 01:03:40.367269 13730 webserver.cc:460] Webserver started at http://127.20.81.190:33949/ using document root <none> and password file <none>
I20260810 01:03:40.367957 13730 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:40.368119 13730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:40.368443 13730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:40.372104 13730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "bcbd9695b6384ab6917347bd13b03c58"
format_stamp: "Formatted at 2026-08-10 01:03:40 on dist-test-slave-gp6n"
server_key: "31bc810e954643ca14bc29279e1d266b"
server_key_iv: "f7bc14e0cd4a4c0114bf1da89fb59513"
server_key_version: "encryptionkey@0"
I20260810 01:03:40.372876 13730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "bcbd9695b6384ab6917347bd13b03c58"
format_stamp: "Formatted at 2026-08-10 01:03:40 on dist-test-slave-gp6n"
server_key: "31bc810e954643ca14bc29279e1d266b"
server_key_iv: "f7bc14e0cd4a4c0114bf1da89fb59513"
server_key_version: "encryptionkey@0"
I20260810 01:03:40.378069 13730 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 01:03:40.381492 13747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:40.382885 13730 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 01:03:40.383088 13730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "bcbd9695b6384ab6917347bd13b03c58"
format_stamp: "Formatted at 2026-08-10 01:03:40 on dist-test-slave-gp6n"
server_key: "31bc810e954643ca14bc29279e1d266b"
server_key_iv: "f7bc14e0cd4a4c0114bf1da89fb59513"
server_key_version: "encryptionkey@0"
I20260810 01:03:40.383299 13730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:40.389114 13730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:40.393327 13730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:40.393656 13730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:40.413076 13730 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:35701
I20260810 01:03:40.413110 13804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:35701 every 8 connection(s)
I20260810 01:03:40.414597 13730 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:03:40.418779 13807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:40.423851 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 13730
I20260810 01:03:40.424063 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:03:40.424517 20806 external_mini_cluster.cc:1428] Setting key 1b96ab24bf6c69e03e96030db4370c41
I20260810 01:03:40.431241 13807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58: Bootstrap starting.
I20260810 01:03:40.432730 13811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:40.426325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38109 (local address 127.20.81.190:35701)
0810 01:03:40.427263 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:40.427275 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:40.427320 (+ 45us) server_negotiation.cc:407] Connection header received
0810 01:03:40.429067 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:40.429088 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:40.429443 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:40.429772 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:40.431134 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:40.431141 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:40.431153 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:40.431168 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:40.431230 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:40.431250 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:40.431486 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:40.431594 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 01:03:40.431796 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":407,"threads_started":1}
I20260810 01:03:40.434855 13807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:40.436156 13807 log.cc:824] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:40.436425 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 01:03:40.439313 13807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58: No bootstrap required, opened a new log
I20260810 01:03:40.443955 13811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:40.441153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50645 (local address 127.20.81.190:35701)
0810 01:03:40.441314 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:40.441322 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:40.441911 (+ 589us) server_negotiation.cc:407] Connection header received
0810 01:03:40.442095 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:40.442099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:40.442146 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:40.442222 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:40.443270 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:40.443274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:40.443277 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 01:03:40.443292 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:40.443332 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 01:03:40.443347 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:40.443430 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:40.443600 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 01:03:40.443648 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 01:03:40.449800 13807 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcbd9695b6384ab6917347bd13b03c58" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 35701 } }
I20260810 01:03:40.450233 13807 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:03:40.450356 13807 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcbd9695b6384ab6917347bd13b03c58, State: Initialized, Role: FOLLOWER
I20260810 01:03:40.450987 13807 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcbd9695b6384ab6917347bd13b03c58" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 35701 } }
I20260810 01:03:40.451257 13807 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 01:03:40.451431 13807 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 01:03:40.451639 13807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:40.454480 13807 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcbd9695b6384ab6917347bd13b03c58" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 35701 } }
I20260810 01:03:40.455049 13807 leader_election.cc:302] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcbd9695b6384ab6917347bd13b03c58; no voters:
I20260810 01:03:40.455737 13807 leader_election.cc:288] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 01:03:40.455892 13820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:03:40.456703 13820 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58 [term 1 LEADER]: Becoming Leader. State: Replica: bcbd9695b6384ab6917347bd13b03c58, State: Running, Role: LEADER
I20260810 01:03:40.457265 13820 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcbd9695b6384ab6917347bd13b03c58" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 35701 } }
I20260810 01:03:40.458457 13807 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:40.460709 13821 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcbd9695b6384ab6917347bd13b03c58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcbd9695b6384ab6917347bd13b03c58" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 35701 } } }
I20260810 01:03:40.460553 13822 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcbd9695b6384ab6917347bd13b03c58. Latest consensus state: current_term: 1 leader_uuid: "bcbd9695b6384ab6917347bd13b03c58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcbd9695b6384ab6917347bd13b03c58" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 35701 } } }
I20260810 01:03:40.461081 13821 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:40.461143 13822 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:40.466012 13827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:03:40.471925 13827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:03:40.481290 13827 catalog_manager.cc:1347] Generated new cluster ID: a6729daa7c4d4c75b8b6c32d3269b171
I20260810 01:03:40.481482 13827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:03:40.505263 13827 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:03:40.505788 13827 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:03:40.514518 13827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bcbd9695b6384ab6917347bd13b03c58: Generated new TSK 0
I20260810 01:03:40.515507 13827 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:03:40.530437 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 13730
W20260810 01:03:40.532052 13799 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38109
W20260810 01:03:40.532823 20806 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 13730: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 01:03:40.532992 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 13730
2026-08-10T01:03:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2208 ms)
[----------] 18 tests from AuthTokenIssuingTest (39558 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:03:40 dist-test-slave-gp6n krb5kdc[13842](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:03:40 dist-test-slave-gp6n krb5kdc[13842](info): set up 2 sockets
Aug 10 01:03:40 dist-test-slave-gp6n krb5kdc[13842](info): commencing operation
krb5kdc: starting...
W20260810 01:03:41.761107 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.001s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:03:41 dist-test-slave-gp6n krb5kdc[13842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:03:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:03:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 01:03:42.118719 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:44943
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:33361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:44943,127.20.81.189:38799,127.20.81.188:41389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:03:42.478838 13858 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 01:03:42.479254 13858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:42.479394 13858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:42.479539 13858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:42.486771 13858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:42.486912 13858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:42.487022 13858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:42.487111 13858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:42.487236 13858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:42.497879 13858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:33361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:44943,127.20.81.189:38799,127.20.81.188:41389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:44943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:42.499341 13858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:42.500746 13858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:42.512811 13863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:42.512876 13866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:42.512887 13864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:42.514104 13858 server_base.cc:1034] running on GCE node
I20260810 01:03:42.515151 13858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:42.516582 13858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:42.517834 13858 hybrid_clock.cc:648] HybridClock initialized: now 1786323822517770 us; error 40 us; skew 500 ppm
I20260810 01:03:42.518191 13858 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:42 dist-test-slave-gp6n krb5kdc[13842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323822, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:42.521571 13858 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 01:03:42.523195 13858 webserver.cc:460] Webserver started at http://127.20.81.190:42573/ using document root <none> and password file <none>
I20260810 01:03:42.523895 13858 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:42.524026 13858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:42.524348 13858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:42.527469 13858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "055a05b051534448803fafff35488e71"
format_stamp: "Formatted at 2026-08-10 01:03:42 on dist-test-slave-gp6n"
server_key: "d7d5f8e26c012f39e5b3474f799512f0"
server_key_iv: "a08188ba0eba7eb16327de7987abd99c"
server_key_version: "encryptionkey@0"
I20260810 01:03:42.528270 13858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "055a05b051534448803fafff35488e71"
format_stamp: "Formatted at 2026-08-10 01:03:42 on dist-test-slave-gp6n"
server_key: "d7d5f8e26c012f39e5b3474f799512f0"
server_key_iv: "a08188ba0eba7eb16327de7987abd99c"
server_key_version: "encryptionkey@0"
I20260810 01:03:42.533512 13858 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 01:03:42.537137 13874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:42.538475 13858 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 01:03:42.538653 13858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "055a05b051534448803fafff35488e71"
format_stamp: "Formatted at 2026-08-10 01:03:42 on dist-test-slave-gp6n"
server_key: "d7d5f8e26c012f39e5b3474f799512f0"
server_key_iv: "a08188ba0eba7eb16327de7987abd99c"
server_key_version: "encryptionkey@0"
I20260810 01:03:42.538873 13858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:42.551044 13858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:42.555446 13858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:42.555733 13858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:42.573922 13858 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:44943
I20260810 01:03:42.573943 13932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:44943 every 8 connection(s)
I20260810 01:03:42.575346 13858 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:03:42.579159 13933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:42.583860 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 13858
I20260810 01:03:42.584065 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:03:42.584687 20806 external_mini_cluster.cc:1428] Setting key fdffd2c8462b0513cf996d6553bf38da
I20260810 01:03:42.585914 13933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44943 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 01:03:42 dist-test-slave-gp6n krb5kdc[13842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323822, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:03:42.609853 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:42.589291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44943 (local address 127.0.0.1:57980)
0810 01:03:42.590658 (+ 1367us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:42.590701 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:42.592511 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:42.593370 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:42.593377 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:42.594105 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:03:42.594512 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:42.594531 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:42.596096 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:42.596103 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:42.597179 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:42.597191 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:42.598120 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:42.598127 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:42.598312 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:42.599054 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:03:42.599077 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:03:42.601528 (+ 2451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:03:42.605910 (+ 4382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:42.605920 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:42.605928 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:42.606299 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:42.606786 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:42.606789 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:42.606791 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:42.606965 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:42.607978 (+ 1013us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:03:42.607984 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:03:42.608179 (+ 195us) client_negotiation.cc:769] Sending connection context
0810 01:03:42.608377 (+ 198us) client_negotiation.cc:240] Negotiation successful
0810 01:03:42.608743 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1175,"thread_start_us":917,"threads_started":1}
I20260810 01:03:42.609915 13935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:42.589264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57980 (local address 127.20.81.190:44943)
0810 01:03:42.590377 (+ 1113us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:42.590394 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:42.590857 (+ 463us) server_negotiation.cc:407] Connection header received
0810 01:03:42.592902 (+ 2045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:42.592907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:42.593139 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:42.593343 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:42.594727 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:42.595940 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:42.597321 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:42.597877 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:42.598076 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:42.601773 (+ 3697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:42.601786 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:42.601803 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:42.601847 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:42.605765 (+ 3918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:42.606475 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:42.606486 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:42.606492 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:42.606678 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:42.607096 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:42.607103 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:42.607109 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:42.607808 (+ 699us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:42.608040 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:42.608600 (+ 560us) server_negotiation.cc:299] Negotiation successful
0810 01:03:42.608929 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":713,"thread_start_us":459,"threads_started":1}
I20260810 01:03:42.612915 13933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } has no permanent_uuid. Determining permanent_uuid...
I20260810 01:03:42.614066 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:42.613596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:38799 (local address 127.0.0.1:46190)
0810 01:03:42.613835 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:42.613950 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:38799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260810 01:03:42.614449 13878 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.81.189:38799: connect: Connection refused (error 111)
I20260810 01:03:42.616528 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:42.616156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:38799 (local address 127.0.0.1:46196)
0810 01:03:42.616328 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:42.616401 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:38799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 01:03:42.616777 13933 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.189:38799: Network error: Client connection negotiation failed: client connection to 127.20.81.189:38799: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.20.81.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.189.keytab.
Entry for principal kudu/127.20.81.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.189.keytab.
I20260810 01:03:42.661747 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.189:38799
--webserver_interface=127.20.81.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.189.keytab
--principal=kudu/127.20.81.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:33361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:44943,127.20.81.189:38799,127.20.81.188:41389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 01:03:42.666147 13933 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } attempt: 1
I20260810 01:03:42.667395 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:42.666942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:38799 (local address 127.0.0.1:46208)
0810 01:03:42.667142 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:42.667249 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:38799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260810 01:03:42.668409 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:42.668050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:38799 (local address 127.0.0.1:46218)
0810 01:03:42.668210 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:42.668277 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:38799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 01:03:42.668576 13933 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.189:38799: Network error: Client connection negotiation failed: client connection to 127.20.81.189:38799: connect: Connection refused (error 111)
I20260810 01:03:42.736842 13933 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } attempt: 2
I20260810 01:03:42.738193 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:42.737663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:38799 (local address 127.0.0.1:46222)
0810 01:03:42.737946 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:42.738044 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:38799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 01:03:42.739357 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:42.738938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:38799 (local address 127.0.0.1:46228)
0810 01:03:42.739101 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:42.739178 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:38799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260810 01:03:42.739609 13933 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.189:38799: Network error: Client connection negotiation failed: client connection to 127.20.81.189:38799: connect: Connection refused (error 111)
I20260810 01:03:42.830876 13933 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } attempt: 3
I20260810 01:03:42.832171 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:42.831723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:38799 (local address 127.0.0.1:46230)
0810 01:03:42.831927 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:42.832036 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:38799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 01:03:42.833194 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:42.832871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:38799 (local address 127.0.0.1:46238)
0810 01:03:42.833005 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:42.833068 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:38799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 01:03:42.833420 13933 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.189:38799: Network error: Client connection negotiation failed: client connection to 127.20.81.189:38799: connect: Connection refused (error 111)
I20260810 01:03:42.976676 13933 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } attempt: 4
I20260810 01:03:42.978050 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:42.977461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:38799 (local address 127.0.0.1:46242)
0810 01:03:42.977793 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:42.977900 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:38799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199}
I20260810 01:03:42.979084 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:42.978763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:38799 (local address 127.0.0.1:46248)
0810 01:03:42.978887 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:42.978975 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:38799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 01:03:42.979305 13933 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.189:38799: Network error: Client connection negotiation failed: client connection to 127.20.81.189:38799: connect: Connection refused (error 111)
W20260810 01:03:43.013972 13940 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 01:03:43.014299 13940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:43.014369 13940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:43.014547 13940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:43.021938 13940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:43.022032 13940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:43.022091 13940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:43.022141 13940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:43.022203 13940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:43.032176 13940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:33361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:44943,127.20.81.189:38799,127.20.81.188:41389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.189.keytab
--principal=kudu/127.20.81.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.189:38799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.81.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:43.033457 13940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:43.034807 13940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:43.047514 13950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:43.047637 13947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:43.047670 13948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:43.048666 13940 server_base.cc:1034] running on GCE node
I20260810 01:03:43.049535 13940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:43.051054 13940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:43.052311 13940 hybrid_clock.cc:648] HybridClock initialized: now 1786323823052246 us; error 37 us; skew 500 ppm
I20260810 01:03:43.052654 13940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:43 dist-test-slave-gp6n krb5kdc[13842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323823, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:43.056087 13940 init.cc:377] Logged in from keytab as kudu/127.20.81.189@KRBTEST.COM (short username kudu)
I20260810 01:03:43.057672 13940 webserver.cc:460] Webserver started at http://127.20.81.189:41087/ using document root <none> and password file <none>
I20260810 01:03:43.058494 13940 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:43.058632 13940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:43.058975 13940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:43.062026 13940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-1/data/instance:
uuid: "469155b290474b91aff3b84a05f344d3"
format_stamp: "Formatted at 2026-08-10 01:03:43 on dist-test-slave-gp6n"
server_key: "c1b11bf588708bdd0fd13f8a62a90458"
server_key_iv: "ceb17789f8faf596aadbc1990a819fc4"
server_key_version: "encryptionkey@0"
I20260810 01:03:43.062882 13940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-1/wal/instance:
uuid: "469155b290474b91aff3b84a05f344d3"
format_stamp: "Formatted at 2026-08-10 01:03:43 on dist-test-slave-gp6n"
server_key: "c1b11bf588708bdd0fd13f8a62a90458"
server_key_iv: "ceb17789f8faf596aadbc1990a819fc4"
server_key_version: "encryptionkey@0"
I20260810 01:03:43.068069 13940 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260810 01:03:43.071532 13958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:43.072870 13940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 01:03:43.073062 13940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-1/wal
uuid: "469155b290474b91aff3b84a05f344d3"
format_stamp: "Formatted at 2026-08-10 01:03:43 on dist-test-slave-gp6n"
server_key: "c1b11bf588708bdd0fd13f8a62a90458"
server_key_iv: "ceb17789f8faf596aadbc1990a819fc4"
server_key_version: "encryptionkey@0"
I20260810 01:03:43.073288 13940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:43.094554 13940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:43.098771 13940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:43.099063 13940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:43.118219 13940 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.189:38799
I20260810 01:03:43.118238 14022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.189:38799 every 8 connection(s)
I20260810 01:03:43.119691 13940 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-1/data/info.pb
I20260810 01:03:43.123667 14023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:43.126796 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 13940
I20260810 01:03:43.127007 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-1/wal/instance
I20260810 01:03:43.127559 20806 external_mini_cluster.cc:1428] Setting key eb9b31dfa25aa1f725fb15a048832e72
I20260810 01:03:43.133121 14023 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44943 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 01:03:43 dist-test-slave-gp6n krb5kdc[13842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323823, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.189@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:03:43.161566 14025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.136510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57990 (local address 127.20.81.190:44943)
0810 01:03:43.137093 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:43.137102 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:43.138045 (+ 943us) server_negotiation.cc:407] Connection header received
0810 01:03:43.139977 (+ 1932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:43.139986 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:43.140083 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:43.140214 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:43.141737 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.142952 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.145508 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.145973 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.146090 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:43.154587 (+ 8497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:43.154594 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:43.154598 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:43.154641 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:43.157950 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.158937 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.158947 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.158951 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.159094 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.159585 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.159590 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.159592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.159906 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:43.160102 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:43.160880 (+ 778us) server_negotiation.cc:299] Negotiation successful
0810 01:03:43.161133 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":276,"threads_started":1}
WARNING: no policy specified for kudu/127.20.81.188@KRBTEST.COM; defaulting to no policy
I20260810 01:03:43.162827 14027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.137076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44943 (local address 127.0.0.1:57990)
0810 01:03:43.137853 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.137912 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:43.139701 (+ 1789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:43.140405 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:43.140412 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:43.140949 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:03:43.141524 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:43.141543 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.143145 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.143153 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:43.145315 (+ 2162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:43.145333 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.146222 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.146232 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:43.146600 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:43.151174 (+ 4574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:03:43.151217 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:03:43.154261 (+ 3044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:03:43.158147 (+ 3886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:43.158157 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:43.158176 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:43.158797 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:43.159226 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:43.159232 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:43.159236 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:43.159438 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:43.160056 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:03:43.160065 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:03:43.160470 (+ 405us) client_negotiation.cc:769] Sending connection context
0810 01:03:43.160767 (+ 297us) client_negotiation.cc:240] Negotiation successful
0810 01:03:43.161395 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":340,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.188@KRBTEST.COM" created.
I20260810 01:03:43.164891 14023 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 01:03:43 dist-test-slave-gp6n krb5kdc[13842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323823, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.189@KRBTEST.COM for kudu/127.20.81.189@KRBTEST.COM
I20260810 01:03:43.181532 14027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.165645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:38799 (local address 127.0.0.1:46252)
0810 01:03:43.165948 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.165971 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:43.166094 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:43.167381 (+ 1287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:43.167384 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:43.167746 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:03:43.167898 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:43.167906 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.170137 (+ 2231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.170141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:43.171094 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:43.171106 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.172426 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.172431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:43.172640 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:43.173260 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:03:43.173277 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:03:43.175410 (+ 2133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:03:43.178945 (+ 3535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:43.178953 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:43.178959 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:43.179318 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:43.179683 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:43.179689 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:43.179692 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:43.179843 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:43.180465 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:03:43.180471 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:03:43.180634 (+ 163us) client_negotiation.cc:769] Sending connection context
0810 01:03:43.180763 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 01:03:43.181081 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":41}
I20260810 01:03:43.182570 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.165997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46252 (local address 127.20.81.189:38799)
0810 01:03:43.166789 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:43.166800 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:43.166837 (+ 37us) server_negotiation.cc:407] Connection header received
0810 01:03:43.166900 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:43.166912 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:43.167193 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:43.167370 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:43.168080 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.169965 (+ 1885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.171239 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.172257 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.172427 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:43.175623 (+ 3196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:43.175634 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:43.175648 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:43.175680 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:43.178822 (+ 3142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.179451 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.179464 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.179467 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.179576 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.179946 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.179953 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.179956 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.180309 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:43.181887 (+ 1578us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:43.182055 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 01:03:43.182238 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":564,"thread_start_us":415,"threads_started":1}
I20260810 01:03:43.184803 14023 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } has no permanent_uuid. Determining permanent_uuid...
I20260810 01:03:43.186154 14027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.185504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:41389 (local address 127.0.0.1:53060)
0810 01:03:43.185763 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.185941 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260810 01:03:43.186463 13964 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
I20260810 01:03:43.188153 14027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.187684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:41389 (local address 127.0.0.1:53062)
0810 01:03:43.187887 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.188003 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260810 01:03:43.188380 14023 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:41389: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.188.keytab.
Entry for principal kudu/127.20.81.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.188.keytab.
I20260810 01:03:43.210191 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.188:41389
--webserver_interface=127.20.81.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.188.keytab
--principal=kudu/127.20.81.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:33361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:44943,127.20.81.189:38799,127.20.81.188:41389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 01:03:43.237748 14023 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } attempt: 1
I20260810 01:03:43.239279 14027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.238631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:41389 (local address 127.0.0.1:53070)
0810 01:03:43.238906 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.239060 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260810 01:03:43.240413 14027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.239950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:41389 (local address 127.0.0.1:53078)
0810 01:03:43.240151 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.240255 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260810 01:03:43.240718 14023 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:41389: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
I20260810 01:03:43.278528 13933 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } attempt: 5
Aug 10 01:03:43 dist-test-slave-gp6n krb5kdc[13842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323822, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for kudu/127.20.81.189@KRBTEST.COM
I20260810 01:03:43.291836 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.279442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46262 (local address 127.20.81.189:38799)
0810 01:03:43.279635 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:43.279644 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:43.279667 (+ 23us) server_negotiation.cc:407] Connection header received
0810 01:03:43.279903 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:43.279912 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:43.279978 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:43.280081 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:43.280887 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.281608 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.282861 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.283291 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.283439 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:43.286541 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:43.286546 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:43.286550 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:43.286584 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:43.289192 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.289895 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.289900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.289904 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.290025 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.290395 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.290424 (+ 29us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.290427 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.290708 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:43.290868 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:43.291282 (+ 414us) server_negotiation.cc:299] Negotiation successful
0810 01:03:43.291503 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 01:03:43.291851 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.279329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:38799 (local address 127.0.0.1:46262)
0810 01:03:43.279567 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.279588 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:43.279734 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:43.280120 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:43.280124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:43.280557 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:03:43.280730 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:43.280739 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.281743 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.281749 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:43.282702 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:43.282714 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.283444 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.283449 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:43.283675 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:43.284315 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:03:43.284333 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:03:43.286320 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:03:43.289382 (+ 3062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:43.289387 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:43.289391 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:43.289769 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:43.290136 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:43.290140 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:43.290144 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:43.290286 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:43.290838 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:03:43.290843 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:03:43.291002 (+ 159us) client_negotiation.cc:769] Sending connection context
0810 01:03:43.291172 (+ 170us) client_negotiation.cc:240] Negotiation successful
0810 01:03:43.291471 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260810 01:03:43.293154 13933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } has no permanent_uuid. Determining permanent_uuid...
I20260810 01:03:43.294278 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.293854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:41389 (local address 127.0.0.1:53082)
0810 01:03:43.294044 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.294127 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260810 01:03:43.295339 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.294963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:41389 (local address 127.0.0.1:53096)
0810 01:03:43.295129 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.295212 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260810 01:03:43.295563 13933 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:41389: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
I20260810 01:03:43.308965 14023 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } attempt: 2
I20260810 01:03:43.310170 14027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.309603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:41389 (local address 127.0.0.1:53104)
0810 01:03:43.309821 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.309937 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260810 01:03:43.311398 14027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.310938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:41389 (local address 127.0.0.1:53112)
0810 01:03:43.311135 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.311237 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260810 01:03:43.311647 14023 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:41389: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
I20260810 01:03:43.346789 13933 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } attempt: 1
I20260810 01:03:43.347957 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.347474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:41389 (local address 127.0.0.1:53120)
0810 01:03:43.347683 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.347783 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260810 01:03:43.349045 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.348728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:41389 (local address 127.0.0.1:53128)
0810 01:03:43.348881 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.348945 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 01:03:43.349323 13933 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:41389: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
I20260810 01:03:43.402913 14023 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } attempt: 3
I20260810 01:03:43.404338 14027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.403766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:41389 (local address 127.0.0.1:53144)
0810 01:03:43.404017 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.404134 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260810 01:03:43.405598 14027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.405100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:41389 (local address 127.0.0.1:53160)
0810 01:03:43.405343 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.405451 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
W20260810 01:03:43.405894 14023 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:41389: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
I20260810 01:03:43.417577 13933 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } attempt: 2
I20260810 01:03:43.418874 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.418287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:41389 (local address 127.0.0.1:53166)
0810 01:03:43.418594 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.418699 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260810 01:03:43.420007 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.419642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:41389 (local address 127.0.0.1:53170)
0810 01:03:43.419809 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.419890 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260810 01:03:43.420217 13933 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:41389: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
I20260810 01:03:43.526525 13933 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } attempt: 3
I20260810 01:03:43.527824 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.527252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:41389 (local address 127.0.0.1:53186)
0810 01:03:43.527496 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.527608 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260810 01:03:43.529132 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.528679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:41389 (local address 127.0.0.1:53194)
0810 01:03:43.528872 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.528980 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 01:03:43.529392 13933 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:41389: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
I20260810 01:03:43.549156 14023 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } attempt: 4
I20260810 01:03:43.550513 14027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.549879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:41389 (local address 127.0.0.1:53206)
0810 01:03:43.550161 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.550272 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260810 01:03:43.551623 14027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.551239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:41389 (local address 127.0.0.1:53212)
0810 01:03:43.551413 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.551499 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 01:03:43.551909 14023 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:41389: Network error: Client connection negotiation failed: client connection to 127.20.81.188:41389: connect: Connection refused (error 111)
W20260810 01:03:43.569558 14033 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 01:03:43.569954 14033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:43.570084 14033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:43.570219 14033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:43.580823 14033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:43.580958 14033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:43.581043 14033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:43.581120 14033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:43.581209 14033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:43.592655 14033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:33361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:44943,127.20.81.189:38799,127.20.81.188:41389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.188.keytab
--principal=kudu/127.20.81.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.188:41389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.81.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:43.594024 14033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:43.595422 14033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:43.607628 14041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:43.607556 14042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:43.607594 14045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:43.608552 14033 server_base.cc:1034] running on GCE node
I20260810 01:03:43.609397 14033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:43.610917 14033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:43.612178 14033 hybrid_clock.cc:648] HybridClock initialized: now 1786323823612112 us; error 35 us; skew 500 ppm
I20260810 01:03:43.612514 14033 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:43 dist-test-slave-gp6n krb5kdc[13842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323823, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:43.615746 14033 init.cc:377] Logged in from keytab as kudu/127.20.81.188@KRBTEST.COM (short username kudu)
I20260810 01:03:43.617565 14033 webserver.cc:460] Webserver started at http://127.20.81.188:43693/ using document root <none> and password file <none>
I20260810 01:03:43.618300 14033 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:43.618392 14033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:43.618742 14033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:43.621713 14033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-2/data/instance:
uuid: "2697dff2253e43cf983c3029a023e7fa"
format_stamp: "Formatted at 2026-08-10 01:03:43 on dist-test-slave-gp6n"
server_key: "0b2db3b4c8a999c7be2328ff0f3991d3"
server_key_iv: "0cf0990857a11241bf1375c85fd76ed4"
server_key_version: "encryptionkey@0"
I20260810 01:03:43.622519 14033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-2/wal/instance:
uuid: "2697dff2253e43cf983c3029a023e7fa"
format_stamp: "Formatted at 2026-08-10 01:03:43 on dist-test-slave-gp6n"
server_key: "0b2db3b4c8a999c7be2328ff0f3991d3"
server_key_iv: "0cf0990857a11241bf1375c85fd76ed4"
server_key_version: "encryptionkey@0"
I20260810 01:03:43.627897 14033 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 01:03:43.631646 14054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:43.633082 14033 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.001s
I20260810 01:03:43.633330 14033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-2/wal
uuid: "2697dff2253e43cf983c3029a023e7fa"
format_stamp: "Formatted at 2026-08-10 01:03:43 on dist-test-slave-gp6n"
server_key: "0b2db3b4c8a999c7be2328ff0f3991d3"
server_key_iv: "0cf0990857a11241bf1375c85fd76ed4"
server_key_version: "encryptionkey@0"
I20260810 01:03:43.633590 14033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:43.648720 14033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:43.652902 14033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:43.653141 14033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:43.672988 14033 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.188:41389
I20260810 01:03:43.673017 14115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.188:41389 every 8 connection(s)
I20260810 01:03:43.674499 14033 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-2/data/info.pb
I20260810 01:03:43.674865 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 14033
I20260810 01:03:43.675069 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323445691443-20806-0/minicluster-data/master-2/wal/instance
I20260810 01:03:43.675757 20806 external_mini_cluster.cc:1428] Setting key 2107999ee283b3ed940902d52513bbf9
I20260810 01:03:43.679353 14117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:43.687750 14117 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44943 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 01:03:43 dist-test-slave-gp6n krb5kdc[13842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.188@KRBTEST.COM
Aug 10 01:03:43 dist-test-slave-gp6n krb5kdc[13842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323823, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.188@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:03:43.699815 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.677708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53228 (local address 127.20.81.188:41389)
0810 01:03:43.678555 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:43.678568 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:43.678610 (+ 42us) server_negotiation.cc:407] Connection header received
0810 01:03:43.680300 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:43.680321 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:43.680685 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:43.681013 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:43.682843 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.684159 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.686027 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.686804 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.686951 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:43.692435 (+ 5484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:43.692443 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:43.692457 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:43.692492 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:43.695594 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.696303 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.696309 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.696315 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.696473 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.696853 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.696863 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.696869 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.697515 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:43.697893 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:43.698319 (+ 426us) server_negotiation.cc:299] Negotiation successful
0810 01:03:43.698750 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":372,"threads_started":1}
I20260810 01:03:43.705739 14122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.691055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:44943 (local address 127.0.0.1:58004)
0810 01:03:43.692028 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.692063 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:43.692251 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:43.692659 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:43.692668 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:43.693310 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:03:43.693611 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:43.693621 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.694744 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.694748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:43.695842 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:43.695854 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.696623 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.696630 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:43.696796 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:43.697363 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:03:43.697379 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:03:43.699648 (+ 2269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:03:43.703281 (+ 3633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:43.703290 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:43.703295 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:43.703812 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:43.704176 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:43.704181 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:43.704185 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:43.704356 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:43.704879 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:03:43.704886 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:03:43.705099 (+ 213us) client_negotiation.cc:769] Sending connection context
0810 01:03:43.705244 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 01:03:43.705470 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"thread_start_us":630,"threads_started":1}
I20260810 01:03:43.706037 14121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.691085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58004 (local address 127.20.81.190:44943)
0810 01:03:43.691714 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:43.691722 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:43.692178 (+ 456us) server_negotiation.cc:407] Connection header received
0810 01:03:43.692393 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:43.692399 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:43.692489 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:43.692606 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:43.693766 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.694580 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.696002 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.696432 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.696581 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:43.699846 (+ 3265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:43.699851 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:43.699854 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:43.699886 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:43.703152 (+ 3266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.703937 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.703941 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.703943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.704062 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.704501 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.704504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.704506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.704745 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:43.704861 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:43.705392 (+ 531us) server_negotiation.cc:299] Negotiation successful
0810 01:03:43.705630 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":339,"threads_started":1}
I20260810 01:03:43.706620 13933 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } attempt: 4
I20260810 01:03:43.707681 14117 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 01:03:43 dist-test-slave-gp6n krb5kdc[13842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
Aug 10 01:03:43 dist-test-slave-gp6n krb5kdc[13842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323822, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for kudu/127.20.81.188@KRBTEST.COM
Aug 10 01:03:43 dist-test-slave-gp6n krb5kdc[13842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323823, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.188@KRBTEST.COM for kudu/127.20.81.189@KRBTEST.COM
I20260810 01:03:43.721503 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.702670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58012 (local address 127.20.81.190:44943)
0810 01:03:43.703357 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:43.703361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:43.703380 (+ 19us) server_negotiation.cc:407] Connection header received
0810 01:03:43.703414 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:43.703417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:43.703467 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:43.703544 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:43.705056 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.706244 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.708053 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.708793 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.708920 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:43.714238 (+ 5318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:43.714244 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:43.714249 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:43.714289 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:43.718257 (+ 3968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.719212 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.719217 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.719221 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.719388 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.719835 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.719840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.719843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.720197 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:43.720517 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:43.720793 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 01:03:43.721034 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":308,"threads_started":1}
I20260810 01:03:43.723431 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.707160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:41389 (local address 127.0.0.1:53234)
0810 01:03:43.707397 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.707416 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:43.707508 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:43.707837 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:43.707841 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:43.708353 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:03:43.708561 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:43.708575 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.710385 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.710390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:43.711433 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:43.711463 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.712510 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.712515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:43.712729 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:43.713581 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:03:43.713603 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:03:43.716293 (+ 2690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:03:43.720211 (+ 3918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:43.720217 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:43.720221 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:43.720791 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:43.721232 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:43.721240 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:43.721243 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:43.721432 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:43.722502 (+ 1070us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:03:43.722510 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:03:43.722677 (+ 167us) client_negotiation.cc:769] Sending connection context
0810 01:03:43.722797 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 01:03:43.723038 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132}
I20260810 01:03:43.724162 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.708494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46266 (local address 127.20.81.189:38799)
0810 01:03:43.708836 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:43.708842 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:43.708859 (+ 17us) server_negotiation.cc:407] Connection header received
0810 01:03:43.708913 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:43.708918 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:43.708996 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:43.709108 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:43.709997 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.711175 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.712733 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.713715 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.713857 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:43.717717 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:43.717722 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:43.717725 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:43.717759 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:43.720893 (+ 3134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.721756 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.721762 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.721765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.721902 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.722352 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.722357 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.722360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.722707 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:43.722864 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:43.723696 (+ 832us) server_negotiation.cc:299] Negotiation successful
0810 01:03:43.723884 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208}
I20260810 01:03:43.724855 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.707374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53234 (local address 127.20.81.188:41389)
0810 01:03:43.707541 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:43.707546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:43.707567 (+ 21us) server_negotiation.cc:407] Connection header received
0810 01:03:43.707617 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:43.707623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:43.707691 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:43.707788 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:43.709297 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.710258 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.711598 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.712359 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.712499 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:43.716482 (+ 3983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:43.716490 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:43.716497 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:43.716545 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:43.720083 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.720938 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.720945 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.720949 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.721086 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.721588 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.721593 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.721936 (+ 343us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.722322 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:43.723871 (+ 1549us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:43.724022 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 01:03:43.724325 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":57}
I20260810 01:03:43.724953 14122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.708314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:38799 (local address 127.0.0.1:46266)
0810 01:03:43.708536 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.708552 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:43.708641 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:43.709128 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:43.709132 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:43.709644 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:03:43.709824 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:43.709835 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.711346 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.711354 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:43.712572 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:43.712591 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.714068 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.714074 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:43.714270 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:43.714877 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:03:43.714904 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:03:43.717462 (+ 2558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:03:43.721046 (+ 3584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:43.721050 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:43.721054 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:43.721586 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:43.722062 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:43.722066 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:43.722070 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:43.722225 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:43.722835 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:03:43.722839 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:03:43.723394 (+ 555us) client_negotiation.cc:769] Sending connection context
0810 01:03:43.724504 (+ 1110us) client_negotiation.cc:240] Negotiation successful
0810 01:03:43.724704 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125}
I20260810 01:03:43.725857 14117 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } has no permanent_uuid. Determining permanent_uuid...
I20260810 01:03:43.731072 13933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71: Bootstrap starting.
I20260810 01:03:43.734220 13933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71: Neither blocks nor log segments found. Creating new log.
Aug 10 01:03:43 dist-test-slave-gp6n krb5kdc[13842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.189@KRBTEST.COM
I20260810 01:03:43.735555 13933 log.cc:824] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:43.739050 13933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71: No bootstrap required, opened a new log
Aug 10 01:03:43 dist-test-slave-gp6n krb5kdc[13842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323823, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.188@KRBTEST.COM for kudu/127.20.81.188@KRBTEST.COM
I20260810 01:03:43.742444 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.723316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46270 (local address 127.20.81.189:38799)
0810 01:03:43.724468 (+ 1152us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:43.724472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:43.726722 (+ 2250us) server_negotiation.cc:407] Connection header received
0810 01:03:43.726955 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:43.726960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:43.727025 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:43.727075 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:43.728341 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.729334 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.730654 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.731075 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.731209 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:43.735492 (+ 4283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:43.735499 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:43.735503 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:43.735544 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:43.739050 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.739972 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.739978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.739981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.740151 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.740637 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.740643 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.740646 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.740996 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:43.741271 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:43.741661 (+ 390us) server_negotiation.cc:299] Negotiation successful
0810 01:03:43.741903 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1047,"thread_start_us":5147,"threads_started":1}
I20260810 01:03:43.747506 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 01:03:43.749692 14122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.726544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:41389 (local address 127.0.0.1:53240)
0810 01:03:43.726754 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.726771 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:43.726850 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:43.734869 (+ 8019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:43.734874 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:43.735349 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:03:43.735527 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:43.735535 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.736771 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.736780 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:43.738161 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:43.738179 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.739401 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.739409 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:43.739611 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:43.740278 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:03:43.740302 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:03:43.742903 (+ 2601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:03:43.746907 (+ 4004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:43.746912 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:43.746916 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:43.747413 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:43.747788 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:43.747793 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:43.747797 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:43.747935 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:43.748688 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:03:43.748693 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:03:43.748843 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 01:03:43.748994 (+ 151us) client_negotiation.cc:240] Negotiation successful
0810 01:03:43.749314 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260810 01:03:43.750294 13906 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "055a05b051534448803fafff35488e71"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58012
I20260810 01:03:43.750531 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.730579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53240 (local address 127.20.81.188:41389)
0810 01:03:43.734521 (+ 3942us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:43.734527 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:43.734548 (+ 21us) server_negotiation.cc:407] Connection header received
0810 01:03:43.734606 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:43.734612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:43.734693 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:43.734803 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:43.735655 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.736619 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.738299 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.739060 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.739177 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:43.743153 (+ 3976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:43.743158 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:43.743164 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:43.743202 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:43.746769 (+ 3567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.747534 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.747538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.747541 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.747677 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.748045 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.748050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.748055 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.748475 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:43.748625 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:43.749976 (+ 1351us) server_negotiation.cc:299] Negotiation successful
0810 01:03:43.750178 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3795}
I20260810 01:03:43.750540 13933 raft_consensus.cc:348] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a05b051534448803fafff35488e71" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44943 } } peers { permanent_uuid: "469155b290474b91aff3b84a05f344d3" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } } peers { permanent_uuid: "2697dff2253e43cf983c3029a023e7fa" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } }
I20260810 01:03:43.750933 13933 raft_consensus.cc:729] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 055a05b051534448803fafff35488e71, State: Initialized, Role: FOLLOWER
I20260810 01:03:43.751498 13933 consensus_queue.cc:260] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a05b051534448803fafff35488e71" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44943 } } peers { permanent_uuid: "469155b290474b91aff3b84a05f344d3" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } } peers { permanent_uuid: "2697dff2253e43cf983c3029a023e7fa" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } }
I20260810 01:03:43.752527 14126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a05b051534448803fafff35488e71" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44943 } } peers { permanent_uuid: "469155b290474b91aff3b84a05f344d3" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } } peers { permanent_uuid: "2697dff2253e43cf983c3029a023e7fa" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } } }
I20260810 01:03:43.752971 14126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:03:43.753926 13933 sys_catalog.cc:564] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:43.753980 13906 raft_consensus.cc:482] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 01:03:43.754212 13906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:43.756341 14117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2697dff2253e43cf983c3029a023e7fa: Bootstrap starting.
I20260810 01:03:43.757862 13906 raft_consensus.cc:504] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a05b051534448803fafff35488e71" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44943 } } peers { permanent_uuid: "469155b290474b91aff3b84a05f344d3" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } } peers { permanent_uuid: "2697dff2253e43cf983c3029a023e7fa" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } }
I20260810 01:03:43.759474 13906 leader_election.cc:288] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71 [CANDIDATE]: Term 1 election: Requested vote from peers 469155b290474b91aff3b84a05f344d3 (127.20.81.189:38799), 2697dff2253e43cf983c3029a023e7fa (127.20.81.188:41389)
I20260810 01:03:43.760632 14117 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2697dff2253e43cf983c3029a023e7fa: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:43.762213 14117 log.cc:824] T 00000000000000000000000000000000 P 2697dff2253e43cf983c3029a023e7fa: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:43.772624 14117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2697dff2253e43cf983c3029a023e7fa: No bootstrap required, opened a new log
I20260810 01:03:43.781702 14147 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 01:03:43.794229 14117 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2697dff2253e43cf983c3029a023e7fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a05b051534448803fafff35488e71" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44943 } } peers { permanent_uuid: "469155b290474b91aff3b84a05f344d3" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } } peers { permanent_uuid: "2697dff2253e43cf983c3029a023e7fa" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } }
I20260810 01:03:43.794848 14117 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2697dff2253e43cf983c3029a023e7fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2697dff2253e43cf983c3029a023e7fa, State: Initialized, Role: FOLLOWER
I20260810 01:03:43.787315 13858 master_runner.cc:186] Error getting master registration for 127.20.81.189:38799: OK, instance_id { permanent_uuid: "469155b290474b91aff3b84a05f344d3" instance_seqno: 1786323823112831 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 01:03:43.797813 13858 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:03:43.796064 14117 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2697dff2253e43cf983c3029a023e7fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a05b051534448803fafff35488e71" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44943 } } peers { permanent_uuid: "469155b290474b91aff3b84a05f344d3" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } } peers { permanent_uuid: "2697dff2253e43cf983c3029a023e7fa" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } }
I20260810 01:03:43.799913 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.769964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53248 (local address 127.20.81.188:41389)
0810 01:03:43.770600 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:43.770609 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:43.777143 (+ 6534us) server_negotiation.cc:407] Connection header received
0810 01:03:43.777390 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:43.777398 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:43.777506 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:43.777609 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:43.782042 (+ 4433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.783136 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.785076 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.785770 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.785902 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:43.789474 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:43.789481 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:43.789485 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:43.789525 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:43.793266 (+ 3741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.794055 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.794060 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.794064 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.794197 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.794684 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.794690 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.794693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.795240 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:43.795489 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:43.796065 (+ 576us) server_negotiation.cc:299] Negotiation successful
0810 01:03:43.796312 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475}
I20260810 01:03:43.806226 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.768822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46276 (local address 127.20.81.189:38799)
0810 01:03:43.768990 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:43.768996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:43.782507 (+ 13511us) server_negotiation.cc:407] Connection header received
0810 01:03:43.783405 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:43.783413 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:43.783509 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:43.783584 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:43.787124 (+ 3540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.788265 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.794530 (+ 6265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.795198 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.795333 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:43.798253 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:43.798260 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:43.798265 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:43.798310 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:43.802685 (+ 4375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.803453 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.803461 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.803466 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.803666 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.804121 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.804130 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.804136 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.804632 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:43.804964 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:43.805615 (+ 651us) server_negotiation.cc:299] Negotiation successful
0810 01:03:43.805848 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 01:03:43.806529 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.766975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58020 (local address 127.20.81.190:44943)
0810 01:03:43.767208 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:43.767215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:43.769167 (+ 1952us) server_negotiation.cc:407] Connection header received
0810 01:03:43.770119 (+ 952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:43.770128 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:43.770241 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:43.770341 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:43.777522 (+ 7181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.778665 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.780734 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.781430 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.781571 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:43.785085 (+ 3514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:43.785092 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:43.785096 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:43.785137 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:43.799054 (+ 13917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.802559 (+ 3505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.802604 (+ 45us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.802608 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.803003 (+ 395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.803820 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.803827 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.803832 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.804217 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:43.804474 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:43.805238 (+ 764us) server_negotiation.cc:299] Negotiation successful
0810 01:03:43.805859 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 01:03:43.806429 14149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2697dff2253e43cf983c3029a023e7fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a05b051534448803fafff35488e71" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44943 } } peers { permanent_uuid: "469155b290474b91aff3b84a05f344d3" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } } peers { permanent_uuid: "2697dff2253e43cf983c3029a023e7fa" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } } }
I20260810 01:03:43.806876 14149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2697dff2253e43cf983c3029a023e7fa [sys.catalog]: This master's current role is: FOLLOWER
W20260810 01:03:43.808518 13880 tablet.cc:2369] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:03:43.808797 14117 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2697dff2253e43cf983c3029a023e7fa [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:43.808921 14088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "055a05b051534448803fafff35488e71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2697dff2253e43cf983c3029a023e7fa"
I20260810 01:03:43.809486 14088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2697dff2253e43cf983c3029a023e7fa [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:43.813192 14088 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2697dff2253e43cf983c3029a023e7fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 055a05b051534448803fafff35488e71 in term 1.
I20260810 01:03:43.813789 13878 leader_election.cc:302] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 055a05b051534448803fafff35488e71, 2697dff2253e43cf983c3029a023e7fa; no voters:
I20260810 01:03:43.814235 14126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:03:43.814961 14126 raft_consensus.cc:686] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71 [term 1 LEADER]: Becoming Leader. State: Replica: 055a05b051534448803fafff35488e71, State: Running, Role: LEADER
I20260810 01:03:43.815557 14126 consensus_queue.cc:237] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a05b051534448803fafff35488e71" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44943 } } peers { permanent_uuid: "469155b290474b91aff3b84a05f344d3" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } } peers { permanent_uuid: "2697dff2253e43cf983c3029a023e7fa" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } }
I20260810 01:03:43.830101 14152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 055a05b051534448803fafff35488e71. Latest consensus state: current_term: 1 leader_uuid: "055a05b051534448803fafff35488e71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a05b051534448803fafff35488e71" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44943 } } peers { permanent_uuid: "469155b290474b91aff3b84a05f344d3" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } } peers { permanent_uuid: "2697dff2253e43cf983c3029a023e7fa" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } } }
I20260810 01:03:43.830745 14152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71 [sys.catalog]: This master's current role is: LEADER
W20260810 01:03:43.830991 14163 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2697dff2253e43cf983c3029a023e7fa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 01:03:43.831120 14163 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 01:03:43.835137 14165 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:03:43.841655 14165 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:03:43.851111 14023 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } attempt: 5
I20260810 01:03:43.845091 14033 master_runner.cc:186] Error getting master registration for 127.20.81.189:38799: OK, instance_id { permanent_uuid: "469155b290474b91aff3b84a05f344d3" instance_seqno: 1786323823112831 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 01:03:43.859998 14033 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:03:43.860404 14088 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2697dff2253e43cf983c3029a023e7fa [term 1 FOLLOWER]: Refusing update from remote peer 055a05b051534448803fafff35488e71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:03:43.861368 14126 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2697dff2253e43cf983c3029a023e7fa" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Aug 10 01:03:43 dist-test-slave-gp6n krb5kdc[13842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323823, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.189@KRBTEST.COM for kudu/127.20.81.188@KRBTEST.COM
I20260810 01:03:43.881677 14027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.851826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:41389 (local address 127.0.0.1:53256)
0810 01:03:43.852209 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:43.852231 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:43.852353 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:43.853475 (+ 1122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:43.853483 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:43.854131 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:03:43.854378 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:43.854394 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.855966 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.855970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:43.857464 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:43.857481 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.858853 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.858860 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:43.859132 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:43.861318 (+ 2186us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:03:43.861344 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:03:43.865583 (+ 4239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:03:43.874094 (+ 8511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:43.874105 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:43.874112 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:43.875095 (+ 983us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:43.876557 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:43.876591 (+ 34us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:43.876598 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:43.876948 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:43.879934 (+ 2986us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:03:43.879965 (+ 31us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:03:43.880241 (+ 276us) client_negotiation.cc:769] Sending connection context
0810 01:03:43.880515 (+ 274us) client_negotiation.cc:240] Negotiation successful
0810 01:03:43.881043 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247}
I20260810 01:03:43.881778 14149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2697dff2253e43cf983c3029a023e7fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 055a05b051534448803fafff35488e71. Latest consensus state: current_term: 1 leader_uuid: "055a05b051534448803fafff35488e71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a05b051534448803fafff35488e71" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44943 } } peers { permanent_uuid: "469155b290474b91aff3b84a05f344d3" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } } peers { permanent_uuid: "2697dff2253e43cf983c3029a023e7fa" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } } }
I20260810 01:03:43.882114 14149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2697dff2253e43cf983c3029a023e7fa [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:03:43.885649 14167 mvcc.cc:205] Tried to move back new op lower bound from 7316782382518820864 to 7316782382353739776. Current Snapshot: MvccSnapshot[applied={T|T < 7316782382518820864}]
I20260810 01:03:43.886125 14126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "055a05b051534448803fafff35488e71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a05b051534448803fafff35488e71" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44943 } } peers { permanent_uuid: "469155b290474b91aff3b84a05f344d3" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } } peers { permanent_uuid: "2697dff2253e43cf983c3029a023e7fa" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } } }
I20260810 01:03:43.886451 14126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:43.886584 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:43.852588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53256 (local address 127.20.81.188:41389)
0810 01:03:43.852802 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:43.852810 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:43.852837 (+ 27us) server_negotiation.cc:407] Connection header received
0810 01:03:43.852919 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:43.852927 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:43.853036 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:43.853173 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:43.854675 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.855754 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.857663 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:43.858336 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:43.858598 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:43.865848 (+ 7250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:43.865855 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:43.865860 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:43.865902 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:43.873779 (+ 7877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.875428 (+ 1649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.875434 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.875437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.876108 (+ 671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:43.877492 (+ 1384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:43.877501 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:43.877506 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:43.879512 (+ 2006us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:43.882373 (+ 2861us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:43.885819 (+ 3446us) server_negotiation.cc:299] Negotiation successful
0810 01:03:43.886104 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 01:03:43.893229 14165 catalog_manager.cc:1347] Generated new cluster ID: 3cc198b88df44d839da932f7def08cb5
I20260810 01:03:43.893327 14165 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:03:43.893381 14023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 469155b290474b91aff3b84a05f344d3: Bootstrap starting.
I20260810 01:03:43.897727 14023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 469155b290474b91aff3b84a05f344d3: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:43.899067 14023 log.cc:824] T 00000000000000000000000000000000 P 469155b290474b91aff3b84a05f344d3: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:43.905205 14149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2697dff2253e43cf983c3029a023e7fa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "055a05b051534448803fafff35488e71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a05b051534448803fafff35488e71" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44943 } } peers { permanent_uuid: "469155b290474b91aff3b84a05f344d3" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } } peers { permanent_uuid: "2697dff2253e43cf983c3029a023e7fa" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } } }
I20260810 01:03:43.905506 14149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2697dff2253e43cf983c3029a023e7fa [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:03:43.911278 14165 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:03:43.913077 14165 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:03:43.924011 14023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 469155b290474b91aff3b84a05f344d3: No bootstrap required, opened a new log
I20260810 01:03:43.929414 14165 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71: Generated new TSK 0
I20260810 01:03:43.930616 14165 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:03:43.935537 14023 raft_consensus.cc:348] T 00000000000000000000000000000000 P 469155b290474b91aff3b84a05f344d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a05b051534448803fafff35488e71" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44943 } } peers { permanent_uuid: "469155b290474b91aff3b84a05f344d3" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } } peers { permanent_uuid: "2697dff2253e43cf983c3029a023e7fa" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } }
I20260810 01:03:43.935988 14023 raft_consensus.cc:729] T 00000000000000000000000000000000 P 469155b290474b91aff3b84a05f344d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 469155b290474b91aff3b84a05f344d3, State: Initialized, Role: FOLLOWER
I20260810 01:03:43.936553 14023 consensus_queue.cc:260] T 00000000000000000000000000000000 P 469155b290474b91aff3b84a05f344d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a05b051534448803fafff35488e71" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44943 } } peers { permanent_uuid: "469155b290474b91aff3b84a05f344d3" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } } peers { permanent_uuid: "2697dff2253e43cf983c3029a023e7fa" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } }
I20260810 01:03:43.962000 14023 sys_catalog.cc:564] T 00000000000000000000000000000000 P 469155b290474b91aff3b84a05f344d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:43.962744 13993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 469155b290474b91aff3b84a05f344d3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:43.962064 13994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "055a05b051534448803fafff35488e71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "469155b290474b91aff3b84a05f344d3"
I20260810 01:03:43.959718 14176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 469155b290474b91aff3b84a05f344d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a05b051534448803fafff35488e71" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44943 } } peers { permanent_uuid: "469155b290474b91aff3b84a05f344d3" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } } peers { permanent_uuid: "2697dff2253e43cf983c3029a023e7fa" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } } }
I20260810 01:03:43.966945 14176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 469155b290474b91aff3b84a05f344d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:03:43.972330 13993 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 469155b290474b91aff3b84a05f344d3 [term 1 FOLLOWER]: Refusing update from remote peer 055a05b051534448803fafff35488e71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:03:43.976033 13994 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 469155b290474b91aff3b84a05f344d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 055a05b051534448803fafff35488e71 in term 1.
I20260810 01:03:43.978857 14126 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "469155b290474b91aff3b84a05f344d3" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:03:44.019996 14176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 469155b290474b91aff3b84a05f344d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 055a05b051534448803fafff35488e71. Latest consensus state: current_term: 1 leader_uuid: "055a05b051534448803fafff35488e71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a05b051534448803fafff35488e71" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44943 } } peers { permanent_uuid: "469155b290474b91aff3b84a05f344d3" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } } peers { permanent_uuid: "2697dff2253e43cf983c3029a023e7fa" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } } }
I20260810 01:03:44.020362 14176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 469155b290474b91aff3b84a05f344d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:03:44.020839 14187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 469155b290474b91aff3b84a05f344d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "055a05b051534448803fafff35488e71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a05b051534448803fafff35488e71" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44943 } } peers { permanent_uuid: "469155b290474b91aff3b84a05f344d3" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } } peers { permanent_uuid: "2697dff2253e43cf983c3029a023e7fa" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } } }
I20260810 01:03:44.021309 14187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 469155b290474b91aff3b84a05f344d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:03:44.057494 14199 catalog_manager.cc:1259] Loaded cluster ID: 3cc198b88df44d839da932f7def08cb5
I20260810 01:03:44.057615 14199 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 469155b290474b91aff3b84a05f344d3: loading cluster ID for follower catalog manager: success
I20260810 01:03:44.061533 14199 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 469155b290474b91aff3b84a05f344d3: acquiring CA information for follower catalog manager: success
I20260810 01:03:44.064478 14199 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 469155b290474b91aff3b84a05f344d3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 01:03:44.075897 20806 mini_kdc.cc:348] Time spent kdestroy: real 0.134s user 0.009s sys 0.022s
W20260810 01:03:44.117659 14123 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 01:03:44.106932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58036 (local address 127.20.81.190:44943)
0810 01:03:44.107154 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.107161 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.114089 (+ 6928us) server_negotiation.cc:407] Connection header received
0810 01:03:44.115701 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.115711 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.115834 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.115928 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:44.117304 (+ 1376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58036: BlockingRecv error: recv got EOF from 127.0.0.1:58036 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260810 01:03:44.124812 14029 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 01:03:44.114713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46286 (local address 127.20.81.189:38799)
0810 01:03:44.114992 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.114999 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.121871 (+ 6872us) server_negotiation.cc:407] Connection header received
0810 01:03:44.122780 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.122801 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.122898 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.122979 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:44.124553 (+ 1574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46286: BlockingRecv error: recv got EOF from 127.0.0.1:46286 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260810 01:03:44.130198 14120 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 01:03:44.127462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53268 (local address 127.20.81.188:41389)
0810 01:03:44.127665 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.127671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.128266 (+ 595us) server_negotiation.cc:407] Connection header received
0810 01:03:44.128535 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.128543 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.128660 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.128738 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:44.129919 (+ 1181us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53268: BlockingRecv error: recv got EOF from 127.0.0.1:53268 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 01:03:44.181301 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:44.165000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46292 (local address 127.20.81.189:38799)
0810 01:03:44.165207 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.165213 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.172163 (+ 6950us) server_negotiation.cc:407] Connection header received
0810 01:03:44.172743 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.172750 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.172963 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.173115 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:44.173976 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.175312 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.177764 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.178591 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.178700 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:44.179589 (+ 889us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:44.180383 (+ 794us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:44.180513 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:44.180797 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 01:03:44.180896 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260810 01:03:44.232966 14120 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 01:03:44.217175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53270 (local address 127.20.81.188:41389)
0810 01:03:44.217428 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.217435 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.230393 (+ 12958us) server_negotiation.cc:407] Connection header received
0810 01:03:44.231097 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.231104 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.231279 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.231381 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:44.232670 (+ 1289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53270: BlockingRecv error: recv got EOF from 127.0.0.1:53270 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 01:03:44.298025 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:44.283503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46300 (local address 127.20.81.189:38799)
0810 01:03:44.283736 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.283743 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.290893 (+ 7150us) server_negotiation.cc:407] Connection header received
0810 01:03:44.291245 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.291253 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.291357 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.291436 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:44.292240 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.293435 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.295584 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.296310 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.296418 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:44.297091 (+ 673us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:44.297275 (+ 184us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:44.297347 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:44.297600 (+ 253us) server_negotiation.cc:299] Negotiation successful
0810 01:03:44.297695 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260810 01:03:44.342941 14120 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 01:03:44.333176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53286 (local address 127.20.81.188:41389)
0810 01:03:44.333384 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.333396 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.340515 (+ 7119us) server_negotiation.cc:407] Connection header received
0810 01:03:44.341104 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.341111 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.341218 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.341337 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:44.342656 (+ 1319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53286: BlockingRecv error: recv got EOF from 127.0.0.1:53286 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 01:03:44.395313 14126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "055a05b051534448803fafff35488e71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a05b051534448803fafff35488e71" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 44943 } } peers { permanent_uuid: "469155b290474b91aff3b84a05f344d3" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 38799 } } peers { permanent_uuid: "2697dff2253e43cf983c3029a023e7fa" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 41389 } } }
I20260810 01:03:44.395665 14126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 055a05b051534448803fafff35488e71 [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:44.400278 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:44.382564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46302 (local address 127.20.81.189:38799)
0810 01:03:44.382768 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.382775 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.389791 (+ 7016us) server_negotiation.cc:407] Connection header received
0810 01:03:44.390367 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.390374 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.390497 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.390574 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:44.391390 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.392548 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.397733 (+ 5185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.398462 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.398570 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:44.399275 (+ 705us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:44.399430 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:44.399509 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:44.399778 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 01:03:44.399918 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260810 01:03:44.453146 14120 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 01:03:44.444137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53298 (local address 127.20.81.188:41389)
0810 01:03:44.444334 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.444342 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.451181 (+ 6839us) server_negotiation.cc:407] Connection header received
0810 01:03:44.451647 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.451654 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.451747 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.451818 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:44.452892 (+ 1074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53298: BlockingRecv error: recv got EOF from 127.0.0.1:53298 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 01:03:44.508249 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:44.494357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46318 (local address 127.20.81.189:38799)
0810 01:03:44.494686 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.494693 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.501587 (+ 6894us) server_negotiation.cc:407] Connection header received
0810 01:03:44.501800 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.501807 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.501935 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.502063 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:44.502589 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.503779 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.505991 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.506685 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.506784 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:44.507413 (+ 629us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:44.507552 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:44.507618 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:44.507843 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 01:03:44.507941 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189}
W20260810 01:03:44.529963 14120 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 01:03:44.527784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53312 (local address 127.20.81.188:41389)
0810 01:03:44.528057 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.528063 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.528384 (+ 321us) server_negotiation.cc:407] Connection header received
0810 01:03:44.528612 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.528616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.528702 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.528789 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:44.529728 (+ 939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53312: BlockingRecv error: recv got EOF from 127.0.0.1:53312 (error 108)
Metrics: {"server-negotiator.queue_time_us":153}
I20260810 01:03:44.548499 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:44.542170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46330 (local address 127.20.81.189:38799)
0810 01:03:44.542448 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.542458 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.542895 (+ 437us) server_negotiation.cc:407] Connection header received
0810 01:03:44.543099 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.543103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.543172 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.543317 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:44.543821 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.544574 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.546544 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.547053 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.547153 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:44.547710 (+ 557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:44.547819 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:44.547886 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:44.548082 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 01:03:44.548149 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260810 01:03:44.557425 14120 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 01:03:44.555025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53326 (local address 127.20.81.188:41389)
0810 01:03:44.555302 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.555309 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.555685 (+ 376us) server_negotiation.cc:407] Connection header received
0810 01:03:44.555877 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.555882 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.555959 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.556045 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:44.557106 (+ 1061us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53326: BlockingRecv error: recv got EOF from 127.0.0.1:53326 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
I20260810 01:03:44.584775 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:44.577335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46336 (local address 127.20.81.189:38799)
0810 01:03:44.577567 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.577574 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.578195 (+ 621us) server_negotiation.cc:407] Connection header received
0810 01:03:44.578447 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.578452 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.578551 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.578661 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:44.579188 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.580401 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.582503 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.583202 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.583328 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:44.583923 (+ 595us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:44.584092 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:44.584183 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:44.584384 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 01:03:44.584485 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260810 01:03:44.596418 14120 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 01:03:44.592059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53336 (local address 127.20.81.188:41389)
0810 01:03:44.592275 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.592282 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.594924 (+ 2642us) server_negotiation.cc:407] Connection header received
0810 01:03:44.595102 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.595106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.595181 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.595261 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:44.596170 (+ 909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53336: BlockingRecv error: recv got EOF from 127.0.0.1:53336 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 01:03:44.638167 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:44.632400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46346 (local address 127.20.81.189:38799)
0810 01:03:44.632685 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.632692 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.633081 (+ 389us) server_negotiation.cc:407] Connection header received
0810 01:03:44.633250 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.633256 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.633341 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.633459 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:44.633850 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.634966 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.636294 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.636930 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.637039 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:44.637411 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:44.637543 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:44.637635 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:44.637767 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 01:03:44.637843 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260810 01:03:44.646126 14120 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 01:03:44.644490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53352 (local address 127.20.81.188:41389)
0810 01:03:44.644690 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.644694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.644992 (+ 298us) server_negotiation.cc:407] Connection header received
0810 01:03:44.645138 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.645142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.645211 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.645302 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:44.645934 (+ 632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53352: BlockingRecv error: recv got EOF from 127.0.0.1:53352 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 01:03:44.718986 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:44.713677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46354 (local address 127.20.81.189:38799)
0810 01:03:44.713859 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.713865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.714335 (+ 470us) server_negotiation.cc:407] Connection header received
0810 01:03:44.714565 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.714569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.714650 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.714742 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:44.715148 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.715909 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.717240 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.717744 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.717882 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:44.718300 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:44.718479 (+ 179us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:44.718558 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:44.718709 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 01:03:44.718773 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260810 01:03:44.726905 14120 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 01:03:44.724845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53362 (local address 127.20.81.188:41389)
0810 01:03:44.725030 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.725036 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.725386 (+ 350us) server_negotiation.cc:407] Connection header received
0810 01:03:44.725578 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.725583 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.725667 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.725762 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:44.726652 (+ 890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53362: BlockingRecv error: recv got EOF from 127.0.0.1:53362 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 01:03:44.833305 14163 catalog_manager.cc:1259] Loaded cluster ID: 3cc198b88df44d839da932f7def08cb5
I20260810 01:03:44.833468 14163 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2697dff2253e43cf983c3029a023e7fa: loading cluster ID for follower catalog manager: success
I20260810 01:03:44.836395 14163 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2697dff2253e43cf983c3029a023e7fa: acquiring CA information for follower catalog manager: success
I20260810 01:03:44.838609 14163 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2697dff2253e43cf983c3029a023e7fa: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 01:03:44.863727 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:44.858256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46362 (local address 127.20.81.189:38799)
0810 01:03:44.858496 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.858501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.858788 (+ 287us) server_negotiation.cc:407] Connection header received
0810 01:03:44.859053 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.859058 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.859147 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.859235 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:44.859677 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.860600 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.862008 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.862499 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.862598 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:44.863031 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:44.863143 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:44.863214 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:44.863353 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 01:03:44.863419 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 01:03:44.876372 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:44.870394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53366 (local address 127.20.81.188:41389)
0810 01:03:44.870661 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.870670 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.870950 (+ 280us) server_negotiation.cc:407] Connection header received
0810 01:03:44.871164 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.871177 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.871358 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.871524 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:44.871987 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.872699 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.874021 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.874515 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.874653 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:44.875170 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:44.875712 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:44.875880 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:44.875946 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 01:03:44.876029 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 01:03:44.891944 14314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:44.883166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58050 (local address 127.20.81.190:44943)
0810 01:03:44.883669 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.883677 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.883751 (+ 74us) server_negotiation.cc:407] Connection header received
0810 01:03:44.884064 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.884073 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.884346 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.884546 (+ 200us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:44.885061 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.886319 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.888221 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.888666 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.888802 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:44.889391 (+ 589us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:44.889750 (+ 359us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:44.889879 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:44.891463 (+ 1584us) server_negotiation.cc:299] Negotiation successful
0810 01:03:44.891579 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":261,"threads_started":1}
I20260810 01:03:44.892069 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:44.883974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46366 (local address 127.20.81.189:38799)
0810 01:03:44.884181 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.884186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.885086 (+ 900us) server_negotiation.cc:407] Connection header received
0810 01:03:44.885372 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.885378 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.885470 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.885568 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:44.886570 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.887335 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.889966 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.890577 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.890676 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:44.891221 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:44.891415 (+ 194us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:44.891516 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:44.891706 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 01:03:44.891793 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 01:03:44.892693 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:44.884634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53372 (local address 127.20.81.188:41389)
0810 01:03:44.884860 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:44.884866 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:44.885432 (+ 566us) server_negotiation.cc:407] Connection header received
0810 01:03:44.885563 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:44.885568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:44.885643 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:44.885776 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:44.886366 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.887523 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.890524 (+ 3001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:44.891273 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:44.891376 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:44.891965 (+ 589us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:44.892094 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:44.892167 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:44.892413 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 01:03:44.892489 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 01:03:44.897431 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 13858
I20260810 01:03:44.898734 13928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:03:45.011931 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 13858
W20260810 01:03:45.038481 13964 connection.cc:535] server connection from 127.0.0.1:46262 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 01:03:45.038939 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 13940
I20260810 01:03:45.040109 14018 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:03:45.148752 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 13940
I20260810 01:03:45.174206 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 14033
I20260810 01:03:45.175413 14111 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:03:45.285439 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 14033
2026-08-10T01:03:45Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4758 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 01:03:45 dist-test-slave-gp6n krb5kdc[14322](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 01:03:45 dist-test-slave-gp6n krb5kdc[14322](info): set up 2 sockets
Aug 10 01:03:45 dist-test-slave-gp6n krb5kdc[14322](info): commencing operation
krb5kdc: starting...
W20260810 01:03:46.516436 20806 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/test-user.keytab.
Aug 10 01:03:46 dist-test-slave-gp6n krb5kdc[14322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T01:03:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:03:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.81.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
Entry for principal kudu/127.20.81.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab.
I20260810 01:03:46.878075 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:43339
--webserver_interface=127.20.81.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:42073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:43339,127.20.81.189:44735,127.20.81.188:37401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 01:03:47.232419 14341 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 01:03:47.232769 14341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:47.232885 14341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:47.233018 14341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:47.240283 14341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:47.240375 14341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:47.240477 14341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:47.240583 14341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:47.240693 14341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:47.251168 14341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:42073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:43339,127.20.81.189:44735,127.20.81.188:37401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.190.keytab
--principal=kudu/127.20.81.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.190:43339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.81.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:47.252526 14341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:47.253891 14341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:47.265748 14346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:47.265784 14349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:47.265826 14347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:47.267006 14341 server_base.cc:1034] running on GCE node
I20260810 01:03:47.267740 14341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:47.269089 14341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:47.270323 14341 hybrid_clock.cc:648] HybridClock initialized: now 1786323827270258 us; error 31 us; skew 500 ppm
I20260810 01:03:47.270725 14341 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:47 dist-test-slave-gp6n krb5kdc[14322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323827, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:47.274046 14341 init.cc:377] Logged in from keytab as kudu/127.20.81.190@KRBTEST.COM (short username kudu)
I20260810 01:03:47.275651 14341 webserver.cc:460] Webserver started at http://127.20.81.190:40253/ using document root <none> and password file <none>
I20260810 01:03:47.276382 14341 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:47.276511 14341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:47.276866 14341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:47.280054 14341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-0/data/instance:
uuid: "3d430ff6bcbd404299b1f1d8369ff53f"
format_stamp: "Formatted at 2026-08-10 01:03:47 on dist-test-slave-gp6n"
server_key: "958b68e0d6645f4215604cfdf66deb4a"
server_key_iv: "55f0261899d5442a1b9792a592b066ff"
server_key_version: "encryptionkey@0"
I20260810 01:03:47.280923 14341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-0/wal/instance:
uuid: "3d430ff6bcbd404299b1f1d8369ff53f"
format_stamp: "Formatted at 2026-08-10 01:03:47 on dist-test-slave-gp6n"
server_key: "958b68e0d6645f4215604cfdf66deb4a"
server_key_iv: "55f0261899d5442a1b9792a592b066ff"
server_key_version: "encryptionkey@0"
I20260810 01:03:47.286242 14341 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 01:03:47.290035 14356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:47.291533 14341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:03:47.291719 14341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-0/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-0/wal
uuid: "3d430ff6bcbd404299b1f1d8369ff53f"
format_stamp: "Formatted at 2026-08-10 01:03:47 on dist-test-slave-gp6n"
server_key: "958b68e0d6645f4215604cfdf66deb4a"
server_key_iv: "55f0261899d5442a1b9792a592b066ff"
server_key_version: "encryptionkey@0"
I20260810 01:03:47.291971 14341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:47.304409 14341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:47.308687 14341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:47.308972 14341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:47.327996 14341 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.190:43339
I20260810 01:03:47.328032 14410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.190:43339 every 8 connection(s)
I20260810 01:03:47.329461 14341 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-0/data/info.pb
I20260810 01:03:47.332857 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 14341
I20260810 01:03:47.333065 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-0/wal/instance
I20260810 01:03:47.333729 20806 external_mini_cluster.cc:1428] Setting key bfa142cafc4e75683f4a66d7dc47c160
I20260810 01:03:47.333820 14411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:47.343932 14411 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43339 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 01:03:47 dist-test-slave-gp6n krb5kdc[14322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323827, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:03:47.370558 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:47.348011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:43339 (local address 127.0.0.1:53158)
0810 01:03:47.348939 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:47.348999 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:47.351660 (+ 2661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:47.352662 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:47.352667 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:47.353258 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:03:47.353614 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:47.353632 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:47.356066 (+ 2434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:47.356070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:47.357230 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:47.357242 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:47.358685 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:47.358691 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:47.358815 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:47.359532 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:03:47.359559 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:03:47.361750 (+ 2191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:03:47.366266 (+ 4516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:47.366276 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:47.366285 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:47.366754 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:47.367298 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:47.367302 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:47.367305 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:47.367473 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:47.368521 (+ 1048us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:03:47.368527 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:03:47.368729 (+ 202us) client_negotiation.cc:769] Sending connection context
0810 01:03:47.368959 (+ 230us) client_negotiation.cc:240] Negotiation successful
0810 01:03:47.369370 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":383,"threads_started":1}
I20260810 01:03:47.370584 14415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:47.348011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53158 (local address 127.20.81.190:43339)
0810 01:03:47.349939 (+ 1928us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:47.349946 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:47.349990 (+ 44us) server_negotiation.cc:407] Connection header received
0810 01:03:47.352074 (+ 2084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:47.352085 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:47.352400 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:47.352620 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:47.353844 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:47.355873 (+ 2029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:47.357384 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:47.358445 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:47.358637 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:47.362024 (+ 3387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:47.362039 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:47.362057 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:47.362100 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:47.366138 (+ 4038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:47.366887 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:47.366899 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:47.366905 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:47.367116 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:47.367599 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:47.367605 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:47.367611 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:47.368363 (+ 752us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:47.368596 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:47.369156 (+ 560us) server_negotiation.cc:299] Negotiation successful
0810 01:03:47.369581 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":84,"server-negotiator.queue_time_us":1726,"spinlock_wait_cycles":54528,"thread_start_us":1484,"threads_started":1}
WARNING: no policy specified for kudu/127.20.81.189@KRBTEST.COM; defaulting to no policy
I20260810 01:03:47.373752 14411 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } has no permanent_uuid. Determining permanent_uuid...
I20260810 01:03:47.374846 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:47.374391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:44735 (local address 127.0.0.1:57774)
0810 01:03:47.374587 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:47.374688 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:44735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.189@KRBTEST.COM" created.
W20260810 01:03:47.375100 14360 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.81.189:44735: connect: Connection refused (error 111)
I20260810 01:03:47.376823 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:47.376327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:44735 (local address 127.0.0.1:57784)
0810 01:03:47.376547 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:47.376641 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:44735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260810 01:03:47.377059 14411 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.189:44735: Network error: Client connection negotiation failed: client connection to 127.20.81.189:44735: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.189.keytab.
Entry for principal kudu/127.20.81.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.189.keytab.
I20260810 01:03:47.414255 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.189:44735
--webserver_interface=127.20.81.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.189.keytab
--principal=kudu/127.20.81.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:42073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:43339,127.20.81.189:44735,127.20.81.188:37401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 01:03:47.426491 14411 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } attempt: 1
I20260810 01:03:47.427874 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:47.427271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:44735 (local address 127.0.0.1:57792)
0810 01:03:47.427531 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:47.427674 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:44735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260810 01:03:47.428962 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:47.428615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:44735 (local address 127.0.0.1:57806)
0810 01:03:47.428762 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:47.428832 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:44735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 01:03:47.429255 14411 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.189:44735: Network error: Client connection negotiation failed: client connection to 127.20.81.189:44735: connect: Connection refused (error 111)
I20260810 01:03:47.497519 14411 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } attempt: 2
I20260810 01:03:47.498788 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:47.498274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:44735 (local address 127.0.0.1:57820)
0810 01:03:47.498558 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:47.498649 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:44735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260810 01:03:47.499864 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:47.499534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:44735 (local address 127.0.0.1:57836)
0810 01:03:47.499663 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:47.499741 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:44735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 01:03:47.500150 14411 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.189:44735: Network error: Client connection negotiation failed: client connection to 127.20.81.189:44735: connect: Connection refused (error 111)
I20260810 01:03:47.591431 14411 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } attempt: 3
I20260810 01:03:47.592700 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:47.592233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:44735 (local address 127.0.0.1:57838)
0810 01:03:47.592454 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:47.592560 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:44735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260810 01:03:47.593771 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:47.593438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:44735 (local address 127.0.0.1:57848)
0810 01:03:47.593567 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:47.593635 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:44735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 01:03:47.594144 14411 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.189:44735: Network error: Client connection negotiation failed: client connection to 127.20.81.189:44735: connect: Connection refused (error 111)
I20260810 01:03:47.737402 14411 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } attempt: 4
I20260810 01:03:47.738818 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:47.738239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:44735 (local address 127.0.0.1:57860)
0810 01:03:47.738594 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:47.738684 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:44735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215}
I20260810 01:03:47.739847 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:47.739517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:44735 (local address 127.0.0.1:57876)
0810 01:03:47.739649 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:47.739731 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.189:44735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 01:03:47.740126 14411 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.189:44735: Network error: Client connection negotiation failed: client connection to 127.20.81.189:44735: connect: Connection refused (error 111)
W20260810 01:03:47.762837 14419 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 01:03:47.763197 14419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:47.763319 14419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:47.763466 14419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:47.770771 14419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:47.770859 14419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:47.770972 14419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:47.771041 14419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:47.771127 14419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:47.780797 14419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:42073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:43339,127.20.81.189:44735,127.20.81.188:37401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.189.keytab
--principal=kudu/127.20.81.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.189:44735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.81.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:47.782069 14419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:47.783401 14419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:47.795408 14426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:47.795522 14428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:47.795534 14424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:47.796941 14419 server_base.cc:1034] running on GCE node
I20260810 01:03:47.797703 14419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:47.799158 14419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:47.800386 14419 hybrid_clock.cc:648] HybridClock initialized: now 1786323827800295 us; error 55 us; skew 500 ppm
I20260810 01:03:47.800693 14419 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:47 dist-test-slave-gp6n krb5kdc[14322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323827, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:47.804040 14419 init.cc:377] Logged in from keytab as kudu/127.20.81.189@KRBTEST.COM (short username kudu)
I20260810 01:03:47.805541 14419 webserver.cc:460] Webserver started at http://127.20.81.189:45555/ using document root <none> and password file <none>
I20260810 01:03:47.806253 14419 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:47.806375 14419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:47.806735 14419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:47.809779 14419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-1/data/instance:
uuid: "677cef0a85514ab7b4cdb768ce6e69bc"
format_stamp: "Formatted at 2026-08-10 01:03:47 on dist-test-slave-gp6n"
server_key: "02212cc0d2a44e0edb944b7404f5c9fe"
server_key_iv: "b8d85a4055800cd4e3cfd20f46bf5967"
server_key_version: "encryptionkey@0"
I20260810 01:03:47.810667 14419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-1/wal/instance:
uuid: "677cef0a85514ab7b4cdb768ce6e69bc"
format_stamp: "Formatted at 2026-08-10 01:03:47 on dist-test-slave-gp6n"
server_key: "02212cc0d2a44e0edb944b7404f5c9fe"
server_key_iv: "b8d85a4055800cd4e3cfd20f46bf5967"
server_key_version: "encryptionkey@0"
I20260810 01:03:47.816159 14419 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 01:03:47.820362 14437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:47.821750 14419 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260810 01:03:47.821938 14419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-1/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-1/wal
uuid: "677cef0a85514ab7b4cdb768ce6e69bc"
format_stamp: "Formatted at 2026-08-10 01:03:47 on dist-test-slave-gp6n"
server_key: "02212cc0d2a44e0edb944b7404f5c9fe"
server_key_iv: "b8d85a4055800cd4e3cfd20f46bf5967"
server_key_version: "encryptionkey@0"
I20260810 01:03:47.822161 14419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:47.842828 14419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:47.846805 14419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:47.847090 14419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:47.865890 14419 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.189:44735
I20260810 01:03:47.865908 14496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.189:44735 every 8 connection(s)
I20260810 01:03:47.867422 14419 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-1/data/info.pb
I20260810 01:03:47.869423 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 14419
I20260810 01:03:47.869629 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-1/wal/instance
I20260810 01:03:47.870127 20806 external_mini_cluster.cc:1428] Setting key 280b06eaf88e6424f1be615e2edfe3d4
I20260810 01:03:47.871412 14497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:47.878080 14497 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43339 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 01:03:47 dist-test-slave-gp6n krb5kdc[14322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323827, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.189@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:03:47.904600 14500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:47.881158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53172 (local address 127.20.81.190:43339)
0810 01:03:47.882831 (+ 1673us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:47.882839 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:47.883292 (+ 453us) server_negotiation.cc:407] Connection header received
0810 01:03:47.885178 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:47.885184 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:47.885278 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:47.885402 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:47.887508 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:47.888244 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:47.890796 (+ 2552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:47.891585 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:47.891729 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:47.896918 (+ 5189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:47.896937 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:47.896946 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:47.897020 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:47.900775 (+ 3755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:47.901973 (+ 1198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:47.901979 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:47.901983 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:47.902129 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:47.902664 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:47.902669 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:47.902673 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:47.902999 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:47.903166 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:47.903984 (+ 818us) server_negotiation.cc:299] Negotiation successful
0810 01:03:47.904218 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1513,"thread_start_us":1404,"threads_started":1}
I20260810 01:03:47.905867 14501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:47.881807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:43339 (local address 127.0.0.1:53172)
0810 01:03:47.883128 (+ 1321us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:47.883168 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:47.884896 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:47.885681 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:47.885694 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:47.886539 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:03:47.887276 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:47.887303 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:47.888409 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:47.888417 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:47.890603 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:47.890621 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:47.891849 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:47.891858 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:47.892219 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:47.893041 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:03:47.893071 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:03:47.896601 (+ 3530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:03:47.900934 (+ 4333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:47.900943 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:47.900962 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:47.901819 (+ 857us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:47.902249 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:47.902254 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:47.902258 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:47.902535 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:47.903145 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:03:47.903153 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:03:47.903580 (+ 427us) client_negotiation.cc:769] Sending connection context
0810 01:03:47.903861 (+ 281us) client_negotiation.cc:240] Negotiation successful
0810 01:03:47.904409 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1097,"thread_start_us":886,"threads_started":1}
I20260810 01:03:47.908023 14497 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.81.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.81.188@KRBTEST.COM" created.
Aug 10 01:03:47 dist-test-slave-gp6n krb5kdc[14322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323827, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.189@KRBTEST.COM for kudu/127.20.81.189@KRBTEST.COM
I20260810 01:03:47.922711 14501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:47.908710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:44735 (local address 127.0.0.1:57878)
0810 01:03:47.908885 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:47.908907 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:47.909008 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:47.910210 (+ 1202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:47.910215 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:47.910685 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:03:47.910866 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:47.910881 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:47.912342 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:47.912346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:47.913298 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:47.913310 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:47.914341 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:47.914347 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:47.914556 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:47.915135 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:03:47.915152 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:03:47.917100 (+ 1948us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:03:47.920276 (+ 3176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:47.920283 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:47.920289 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:47.920631 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:47.921026 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:47.921032 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:47.921036 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:47.921168 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:47.921741 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:03:47.921745 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:03:47.921873 (+ 128us) client_negotiation.cc:769] Sending connection context
0810 01:03:47.922019 (+ 146us) client_negotiation.cc:240] Negotiation successful
0810 01:03:47.922289 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 01:03:47.922796 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:47.908981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57878 (local address 127.20.81.189:44735)
0810 01:03:47.909620 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:47.909631 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:47.909667 (+ 36us) server_negotiation.cc:407] Connection header received
0810 01:03:47.909730 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:47.909739 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:47.910015 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:47.910193 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:47.911060 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:47.912183 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:47.913434 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:47.914152 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:47.914297 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:47.917301 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:47.917313 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:47.917326 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:47.917352 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:47.920151 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:47.920796 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:47.920814 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:47.920816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:47.920922 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:47.921267 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:47.921274 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:47.921277 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:47.921597 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:47.921823 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:47.922177 (+ 354us) server_negotiation.cc:299] Negotiation successful
0810 01:03:47.922430 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":289,"threads_started":1}
I20260810 01:03:47.925653 14497 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } has no permanent_uuid. Determining permanent_uuid...
I20260810 01:03:47.927044 14501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:47.926463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:37401 (local address 127.0.0.1:41560)
0810 01:03:47.926684 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:47.926838 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260810 01:03:47.927317 14441 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
I20260810 01:03:47.929096 14501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:47.928637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:37401 (local address 127.0.0.1:41576)
0810 01:03:47.928802 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:47.928940 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 01:03:47.929380 14497 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:37401: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.81.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.188.keytab.
Entry for principal kudu/127.20.81.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.188.keytab.
I20260810 01:03:47.954499 20806 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5NFMaT/build/asan/bin/kudu
/tmp/dist-test-task5NFMaT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.188:37401
--webserver_interface=127.20.81.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.188.keytab
--principal=kudu/127.20.81.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.81.148:42073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.81.190:43339,127.20.81.189:44735,127.20.81.188:37401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 01:03:47.978754 14497 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } attempt: 1
I20260810 01:03:47.980152 14501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:47.979537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:37401 (local address 127.0.0.1:41590)
0810 01:03:47.979795 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:47.979919 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 01:03:47.981339 14501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:47.980862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:37401 (local address 127.0.0.1:41598)
0810 01:03:47.981073 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:47.981186 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 01:03:47.981586 14497 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:37401: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
I20260810 01:03:48.039387 14411 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } attempt: 5
Aug 10 01:03:48 dist-test-slave-gp6n krb5kdc[14322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323827, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for kudu/127.20.81.189@KRBTEST.COM
I20260810 01:03:48.049831 14497 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } attempt: 2
I20260810 01:03:48.050968 14501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.050490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:37401 (local address 127.0.0.1:41602)
0810 01:03:48.050725 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:48.050806 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260810 01:03:48.051954 14501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.051614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:37401 (local address 127.0.0.1:41606)
0810 01:03:48.051755 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:48.051836 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 01:03:48.052259 14497 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:37401: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
I20260810 01:03:48.055248 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.040159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:44735 (local address 127.0.0.1:57886)
0810 01:03:48.040410 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:48.040434 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:48.040575 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:48.040946 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:48.040951 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:48.041492 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:03:48.041752 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:48.041770 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.042861 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.042870 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:48.044465 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:48.044481 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.045204 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.045212 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:48.045439 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:48.046081 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:03:48.046106 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:03:48.048548 (+ 2442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:03:48.052594 (+ 4046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:48.052597 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:48.052600 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:48.052962 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:48.053353 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:48.053356 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:48.053358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:48.053512 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:48.054178 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:03:48.054183 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:03:48.054398 (+ 215us) client_negotiation.cc:769] Sending connection context
0810 01:03:48.054643 (+ 245us) client_negotiation.cc:240] Negotiation successful
0810 01:03:48.055010 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260810 01:03:48.055438 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.040381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57886 (local address 127.20.81.189:44735)
0810 01:03:48.040553 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:48.040557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:48.040577 (+ 20us) server_negotiation.cc:407] Connection header received
0810 01:03:48.040703 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:48.040707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:48.040782 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:48.040867 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:48.041912 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.042714 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.044624 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.045048 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.045147 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:48.048746 (+ 3599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:48.048751 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:48.048754 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:48.048787 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:48.052447 (+ 3660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.053083 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.053089 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.053092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.053243 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.053617 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.053621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.053625 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.054026 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:48.054167 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:48.054755 (+ 588us) server_negotiation.cc:299] Negotiation successful
0810 01:03:48.055067 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 01:03:48.056430 14411 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } has no permanent_uuid. Determining permanent_uuid...
I20260810 01:03:48.057401 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.057028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:37401 (local address 127.0.0.1:41616)
0810 01:03:48.057185 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:48.057268 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 01:03:48.058357 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.058034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:37401 (local address 127.0.0.1:41630)
0810 01:03:48.058181 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:48.058253 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 01:03:48.058591 14411 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:37401: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
I20260810 01:03:48.109867 14411 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } attempt: 1
I20260810 01:03:48.111162 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.110651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:37401 (local address 127.0.0.1:41632)
0810 01:03:48.110867 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:48.110976 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260810 01:03:48.112243 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.111942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:37401 (local address 127.0.0.1:41646)
0810 01:03:48.112074 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:48.112145 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 01:03:48.112534 14411 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:37401: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
I20260810 01:03:48.143514 14497 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } attempt: 3
I20260810 01:03:48.144706 14501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.144213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:37401 (local address 127.0.0.1:41650)
0810 01:03:48.144455 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:48.144545 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 01:03:48.145979 14501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.145545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:37401 (local address 127.0.0.1:41654)
0810 01:03:48.145785 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:48.145864 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
W20260810 01:03:48.146214 14497 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:37401: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
I20260810 01:03:48.180799 14411 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } attempt: 2
I20260810 01:03:48.182124 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.181571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:37401 (local address 127.0.0.1:41660)
0810 01:03:48.181831 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:48.181947 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260810 01:03:48.183549 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.183180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:37401 (local address 127.0.0.1:41672)
0810 01:03:48.183329 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:48.183427 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 01:03:48.183930 14411 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:37401: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
I20260810 01:03:48.289474 14497 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } attempt: 4
I20260810 01:03:48.290174 14411 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } attempt: 3
I20260810 01:03:48.290860 14501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.290212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:37401 (local address 127.0.0.1:41678)
0810 01:03:48.290527 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:48.290644 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 01:03:48.291240 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.290737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:37401 (local address 127.0.0.1:41686)
0810 01:03:48.290926 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:48.291051 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 01:03:48.291980 14501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.291618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:37401 (local address 127.0.0.1:41692)
0810 01:03:48.291783 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:48.291869 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 01:03:48.292273 14497 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:37401: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
I20260810 01:03:48.292523 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.292024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:37401 (local address 127.0.0.1:41694)
0810 01:03:48.292292 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:48.292385 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260810 01:03:48.292698 14411 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.81.188:37401: Network error: Client connection negotiation failed: client connection to 127.20.81.188:37401: connect: Connection refused (error 111)
W20260810 01:03:48.311689 14507 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 01:03:48.312062 14507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:03:48.312186 14507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:03:48.312315 14507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 01:03:48.319801 14507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:03:48.319928 14507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 01:03:48.320048 14507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:03:48.320133 14507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:03:48.320235 14507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:03:48.333242 14507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.81.148:42073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.81.190:43339,127.20.81.189:44735,127.20.81.188:37401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/krb5kdc/kudu_127.20.81.188.keytab
--principal=kudu/127.20.81.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.81.188:37401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.81.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 01:03:48.335034 14507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:03:48.336864 14507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 01:03:48.349570 14513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:48.349601 14514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:03:48.349665 14518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:03:48.350705 14507 server_base.cc:1034] running on GCE node
I20260810 01:03:48.351440 14507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:03:48.352797 14507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:03:48.354046 14507 hybrid_clock.cc:648] HybridClock initialized: now 1786323828353982 us; error 29 us; skew 500 ppm
I20260810 01:03:48.354436 14507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 01:03:48 dist-test-slave-gp6n krb5kdc[14322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323828, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 01:03:48.362177 14507 init.cc:377] Logged in from keytab as kudu/127.20.81.188@KRBTEST.COM (short username kudu)
I20260810 01:03:48.363739 14507 webserver.cc:460] Webserver started at http://127.20.81.188:44709/ using document root <none> and password file <none>
I20260810 01:03:48.364614 14507 fs_manager.cc:359] Metadata directory not provided
I20260810 01:03:48.364784 14507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:03:48.365188 14507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:03:48.368319 14507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-2/data/instance:
uuid: "8b4b685fb46e4a40b742d270de0e7cf9"
format_stamp: "Formatted at 2026-08-10 01:03:48 on dist-test-slave-gp6n"
server_key: "ac650bb2c5a61fcaa2d6dedbadce5e61"
server_key_iv: "333932e33b456ef8e6bf41e8c8a0ee35"
server_key_version: "encryptionkey@0"
I20260810 01:03:48.369148 14507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-2/wal/instance:
uuid: "8b4b685fb46e4a40b742d270de0e7cf9"
format_stamp: "Formatted at 2026-08-10 01:03:48 on dist-test-slave-gp6n"
server_key: "ac650bb2c5a61fcaa2d6dedbadce5e61"
server_key_iv: "333932e33b456ef8e6bf41e8c8a0ee35"
server_key_version: "encryptionkey@0"
I20260810 01:03:48.374356 14507 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260810 01:03:48.377699 14526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:03:48.379165 14507 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:03:48.379359 14507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-2/data,/tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-2/wal
uuid: "8b4b685fb46e4a40b742d270de0e7cf9"
format_stamp: "Formatted at 2026-08-10 01:03:48 on dist-test-slave-gp6n"
server_key: "ac650bb2c5a61fcaa2d6dedbadce5e61"
server_key_iv: "333932e33b456ef8e6bf41e8c8a0ee35"
server_key_version: "encryptionkey@0"
I20260810 01:03:48.379588 14507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:03:48.391872 14507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 01:03:48.396407 14507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:03:48.396699 14507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:03:48.415519 14507 rpc_server.cc:307] RPC server started. Bound to: 127.20.81.188:37401
I20260810 01:03:48.415542 14581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.81.188:37401 every 8 connection(s)
I20260810 01:03:48.416925 14507 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-2/data/info.pb
I20260810 01:03:48.419859 20806 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5NFMaT/build/asan/bin/kudu as pid 14507
I20260810 01:03:48.420068 20806 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5NFMaT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323445691443-20806-0/minicluster-data/master-2/wal/instance
I20260810 01:03:48.420523 20806 external_mini_cluster.cc:1428] Setting key 864f2198ef8c35e088fcf4f187e4744b
I20260810 01:03:48.420833 14582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:03:48.430284 14582 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43339 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 01:03:48 dist-test-slave-gp6n krb5kdc[14322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.188@KRBTEST.COM
Aug 10 01:03:48 dist-test-slave-gp6n krb5kdc[14322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323828, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.188@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
I20260810 01:03:48.445183 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.422279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41700 (local address 127.20.81.188:37401)
0810 01:03:48.423109 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:48.423120 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:48.423160 (+ 40us) server_negotiation.cc:407] Connection header received
0810 01:03:48.425351 (+ 2191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:48.425372 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:48.425809 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:48.426173 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:48.428026 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.429325 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.431212 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.431949 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.432085 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:48.437268 (+ 5183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:48.437275 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:48.437284 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:48.437320 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:48.441144 (+ 3824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.441895 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.441909 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.441920 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.442100 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.442545 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.442548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.442550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.443081 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:48.443438 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:48.443827 (+ 389us) server_negotiation.cc:299] Negotiation successful
0810 01:03:48.444236 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":441,"threads_started":1}
I20260810 01:03:48.449908 14587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.433282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.190:43339 (local address 127.0.0.1:53176)
0810 01:03:48.434522 (+ 1240us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:48.434570 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:48.434750 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:48.435156 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:48.435166 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:48.435849 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:03:48.436123 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:48.436133 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.437617 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.437622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:48.439405 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:48.439426 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.440153 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.440159 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:48.440333 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:48.440904 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:03:48.440929 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:03:48.443358 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:03:48.447073 (+ 3715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:48.447083 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:48.447090 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:48.447628 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:48.448071 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:48.448076 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:48.448080 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:48.448260 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:48.448935 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:03:48.448944 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:03:48.449128 (+ 184us) client_negotiation.cc:769] Sending connection context
0810 01:03:48.449363 (+ 235us) client_negotiation.cc:240] Negotiation successful
0810 01:03:48.449641 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":945,"thread_start_us":785,"threads_started":1}
I20260810 01:03:48.450222 14586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.433331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53176 (local address 127.20.81.190:43339)
0810 01:03:48.433902 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:48.433908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:48.434687 (+ 779us) server_negotiation.cc:407] Connection header received
0810 01:03:48.434893 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:48.434897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:48.434971 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:48.435057 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:48.436296 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.437442 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.439561 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.439965 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.440064 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:48.443552 (+ 3488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:48.443557 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:48.443560 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:48.443592 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:48.446936 (+ 3344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.447770 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.447776 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.447779 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.447938 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.448396 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.448401 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.448404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.448795 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:48.448907 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:48.449538 (+ 631us) server_negotiation.cc:299] Negotiation successful
0810 01:03:48.449767 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":271,"threads_started":1}
I20260810 01:03:48.451555 14582 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 01:03:48 dist-test-slave-gp6n krb5kdc[14322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.190@KRBTEST.COM
Aug 10 01:03:48 dist-test-slave-gp6n krb5kdc[14322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323828, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.188@KRBTEST.COM for kudu/127.20.81.189@KRBTEST.COM
I20260810 01:03:48.463467 14588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.448305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53192 (local address 127.20.81.190:43339)
0810 01:03:48.448962 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:48.448968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:48.448990 (+ 22us) server_negotiation.cc:407] Connection header received
0810 01:03:48.449038 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:48.449043 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:48.449110 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:48.449224 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:48.450802 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.451726 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.453065 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.453497 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.453632 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:48.457677 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:48.457682 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:48.457686 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:48.457715 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:48.460587 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.461237 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.461249 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.461252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.461377 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.461759 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.461764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.461767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.462196 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:48.462495 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:48.462823 (+ 328us) server_negotiation.cc:299] Negotiation successful
0810 01:03:48.463040 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":269,"threads_started":1}
I20260810 01:03:48.463714 14587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.452156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.189:44735 (local address 127.0.0.1:57894)
0810 01:03:48.452306 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:48.452323 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:48.452407 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:48.452860 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:48.452864 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:48.453234 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:03:48.453361 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:48.453368 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.454289 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.454297 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:48.455272 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:48.455284 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.455999 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.456005 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:48.456216 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:48.456695 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:03:48.456716 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:03:48.458634 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:03:48.461419 (+ 2785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:48.461425 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:48.461429 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:48.461772 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:48.462139 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:48.462145 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:48.462149 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:48.462287 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:48.462805 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:03:48.462810 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:03:48.462974 (+ 164us) client_negotiation.cc:769] Sending connection context
0810 01:03:48.463119 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 01:03:48.463381 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 01:03:48.463783 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.452412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57894 (local address 127.20.81.189:44735)
0810 01:03:48.452608 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:48.452613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:48.452627 (+ 14us) server_negotiation.cc:407] Connection header received
0810 01:03:48.452674 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:48.452677 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:48.452733 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:48.452840 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:48.453519 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.454156 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.455416 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.455834 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.455964 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:48.458811 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:48.458816 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:48.458819 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:48.458851 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:48.461288 (+ 2437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.461884 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.461891 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.461894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.462031 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.462391 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.462398 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.462427 (+ 29us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.462679 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:48.462844 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:48.463215 (+ 371us) server_negotiation.cc:299] Negotiation successful
0810 01:03:48.463413 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 01:03:48.464936 14582 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } has no permanent_uuid. Determining permanent_uuid...
I20260810 01:03:48.469894 14411 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } attempt: 4
Aug 10 01:03:48 dist-test-slave-gp6n krb5kdc[14322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323828, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.188@KRBTEST.COM for kudu/127.20.81.188@KRBTEST.COM
Aug 10 01:03:48 dist-test-slave-gp6n krb5kdc[14322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.81.189@KRBTEST.COM
Aug 10 01:03:48 dist-test-slave-gp6n krb5kdc[14322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323827, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.190@KRBTEST.COM for kudu/127.20.81.188@KRBTEST.COM
I20260810 01:03:48.481098 14587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.465626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:37401 (local address 127.0.0.1:41716)
0810 01:03:48.465777 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:48.465791 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:48.465888 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:48.466317 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:48.466323 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:48.466865 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:03:48.467054 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:48.467065 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.468404 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.468414 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:48.469853 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:48.469868 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.471049 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.471058 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:48.471370 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:48.472144 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:03:48.472168 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:03:48.474621 (+ 2453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:03:48.478431 (+ 3810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:48.478436 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:48.478440 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:48.478917 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:48.479344 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:48.479350 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:48.479353 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:48.479520 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:48.480106 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:03:48.480111 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:03:48.480245 (+ 134us) client_negotiation.cc:769] Sending connection context
0810 01:03:48.480373 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 01:03:48.480683 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 01:03:48.481146 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.465832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41716 (local address 127.20.81.188:37401)
0810 01:03:48.466058 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:48.466062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:48.466083 (+ 21us) server_negotiation.cc:407] Connection header received
0810 01:03:48.466116 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:48.466119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:48.466165 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:48.466233 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:48.467201 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.468250 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.470002 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.470685 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.470832 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:48.474819 (+ 3987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:48.474826 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:48.474831 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:48.474870 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:48.478244 (+ 3374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.479053 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.479061 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.479064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.479208 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.479658 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.479663 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.479667 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.479994 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:48.480141 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:48.480462 (+ 321us) server_negotiation.cc:299] Negotiation successful
0810 01:03:48.480836 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":147,"server-negotiator.queue_time_us":144}
I20260810 01:03:48.482756 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.465271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57900 (local address 127.20.81.189:44735)
0810 01:03:48.465421 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:48.465427 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:48.465525 (+ 98us) server_negotiation.cc:407] Connection header received
0810 01:03:48.465706 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:48.465710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:48.465780 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:48.465878 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:48.467397 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.468441 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.470165 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.470816 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.470939 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:48.476077 (+ 5138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:48.476086 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:48.476093 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:48.476135 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:48.479662 (+ 3527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.480543 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.480548 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.480552 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.480716 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.481118 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.481123 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.481127 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.481501 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:48.481767 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:48.482066 (+ 299us) server_negotiation.cc:299] Negotiation successful
0810 01:03:48.482285 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 01:03:48.487329 14582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b4b685fb46e4a40b742d270de0e7cf9: Bootstrap starting.
I20260810 01:03:48.487648 20806 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 01:03:48.488626 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.470482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:37401 (local address 127.0.0.1:41732)
0810 01:03:48.470731 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:48.470749 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:48.470861 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:48.472464 (+ 1603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:48.472470 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:48.473008 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:03:48.473230 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:48.473245 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.474834 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.474839 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:48.476025 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:48.476040 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.476997 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.477002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:48.477204 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:48.477722 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:03:48.477748 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:03:48.480548 (+ 2800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:03:48.485299 (+ 4751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:48.485304 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:48.485308 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:48.486497 (+ 1189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:48.486901 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:48.486906 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:48.486909 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:48.487096 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:48.487727 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:03:48.487732 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:03:48.487902 (+ 170us) client_negotiation.cc:769] Sending connection context
0810 01:03:48.488029 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 01:03:48.488279 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143}
I20260810 01:03:48.489848 14589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.470585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41732 (local address 127.20.81.188:37401)
0810 01:03:48.472144 (+ 1559us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:48.472149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:48.472172 (+ 23us) server_negotiation.cc:407] Connection header received
0810 01:03:48.472217 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:48.472224 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:48.472292 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:48.472397 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:48.473411 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.474732 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.476174 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.476835 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.476975 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:48.481360 (+ 4385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:48.481369 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:48.481379 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:48.481412 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:48.485150 (+ 3738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.486614 (+ 1464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.486619 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.486622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.486776 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.487231 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.487236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.487239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.487554 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:48.489000 (+ 1446us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:48.489135 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 01:03:48.489385 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1423,"thread_start_us":1201,"threads_started":1}
I20260810 01:03:48.490216 14384 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3d430ff6bcbd404299b1f1d8369ff53f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53192
I20260810 01:03:48.495450 14582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b4b685fb46e4a40b742d270de0e7cf9: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:48.496031 14411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f: Bootstrap starting.
I20260810 01:03:48.496703 14582 log.cc:824] T 00000000000000000000000000000000 P 8b4b685fb46e4a40b742d270de0e7cf9: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:48.499542 14582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b4b685fb46e4a40b742d270de0e7cf9: No bootstrap required, opened a new log
I20260810 01:03:48.499815 14411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:48.501051 14411 log.cc:824] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f: Log is configured to *not* fsync() on all Append() calls
I20260810 01:03:48.504176 14411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f: No bootstrap required, opened a new log
I20260810 01:03:48.507221 14582 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b4b685fb46e4a40b742d270de0e7cf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43339 } } peers { permanent_uuid: "677cef0a85514ab7b4cdb768ce6e69bc" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } } peers { permanent_uuid: "8b4b685fb46e4a40b742d270de0e7cf9" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } }
I20260810 01:03:48.507565 14582 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b4b685fb46e4a40b742d270de0e7cf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b4b685fb46e4a40b742d270de0e7cf9, State: Initialized, Role: FOLLOWER
I20260810 01:03:48.508004 14582 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b4b685fb46e4a40b742d270de0e7cf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43339 } } peers { permanent_uuid: "677cef0a85514ab7b4cdb768ce6e69bc" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } } peers { permanent_uuid: "8b4b685fb46e4a40b742d270de0e7cf9" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } }
I20260810 01:03:48.508913 14593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b4b685fb46e4a40b742d270de0e7cf9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43339 } } peers { permanent_uuid: "677cef0a85514ab7b4cdb768ce6e69bc" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } } peers { permanent_uuid: "8b4b685fb46e4a40b742d270de0e7cf9" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } } }
I20260810 01:03:48.509377 14593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b4b685fb46e4a40b742d270de0e7cf9 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:03:48.510069 14582 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b4b685fb46e4a40b742d270de0e7cf9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:48.511584 14411 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43339 } } peers { permanent_uuid: "677cef0a85514ab7b4cdb768ce6e69bc" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } } peers { permanent_uuid: "8b4b685fb46e4a40b742d270de0e7cf9" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } }
I20260810 01:03:48.511997 14411 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d430ff6bcbd404299b1f1d8369ff53f, State: Initialized, Role: FOLLOWER
I20260810 01:03:48.512471 14411 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43339 } } peers { permanent_uuid: "677cef0a85514ab7b4cdb768ce6e69bc" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } } peers { permanent_uuid: "8b4b685fb46e4a40b742d270de0e7cf9" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } }
I20260810 01:03:48.513417 14599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43339 } } peers { permanent_uuid: "677cef0a85514ab7b4cdb768ce6e69bc" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } } peers { permanent_uuid: "8b4b685fb46e4a40b742d270de0e7cf9" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } } }
I20260810 01:03:48.513979 14599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:03:48.514655 14411 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:48.514719 14384 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 01:03:48.514915 14384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:48.518080 14384 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43339 } } peers { permanent_uuid: "677cef0a85514ab7b4cdb768ce6e69bc" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } } peers { permanent_uuid: "8b4b685fb46e4a40b742d270de0e7cf9" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } }
I20260810 01:03:48.521661 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8b4b685fb46e4a40b742d270de0e7cf9"
I20260810 01:03:48.522143 14555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b4b685fb46e4a40b742d270de0e7cf9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:48.526180 14555 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8b4b685fb46e4a40b742d270de0e7cf9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d430ff6bcbd404299b1f1d8369ff53f in term 1.
I20260810 01:03:48.527216 14360 leader_election.cc:302] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d430ff6bcbd404299b1f1d8369ff53f, 8b4b685fb46e4a40b742d270de0e7cf9; no voters:
I20260810 01:03:48.527655 14599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:03:48.526655 14384 leader_election.cc:288] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f [CANDIDATE]: Term 1 election: Requested vote from peers 677cef0a85514ab7b4cdb768ce6e69bc (127.20.81.189:44735), 8b4b685fb46e4a40b742d270de0e7cf9 (127.20.81.188:37401)
I20260810 01:03:48.520045 14507 master_runner.cc:186] Error getting master registration for 127.20.81.190:43339: OK, instance_id { permanent_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" instance_seqno: 1786323827322629 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 01:03:48.531762 14507 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:03:48.542451 14599 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f [term 1 LEADER]: Becoming Leader. State: Replica: 3d430ff6bcbd404299b1f1d8369ff53f, State: Running, Role: LEADER
I20260810 01:03:48.543521 14599 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43339 } } peers { permanent_uuid: "677cef0a85514ab7b4cdb768ce6e69bc" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } } peers { permanent_uuid: "8b4b685fb46e4a40b742d270de0e7cf9" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } }
I20260810 01:03:48.556638 14611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d430ff6bcbd404299b1f1d8369ff53f. Latest consensus state: current_term: 1 leader_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43339 } } peers { permanent_uuid: "677cef0a85514ab7b4cdb768ce6e69bc" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } } peers { permanent_uuid: "8b4b685fb46e4a40b742d270de0e7cf9" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } } }
I20260810 01:03:48.557662 14611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f [sys.catalog]: This master's current role is: LEADER
W20260810 01:03:48.561873 14363 tablet.cc:2369] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 01:03:48.567901 14608 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8b4b685fb46e4a40b742d270de0e7cf9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 01:03:48.568176 14608 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 01:03:48.570235 14623 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 01:03:48.587291 14623 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 01:03:48.591534 14497 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } attempt: 5
I20260810 01:03:48.592240 14588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.534039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53204 (local address 127.20.81.190:43339)
0810 01:03:48.534379 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:48.534389 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:48.541316 (+ 6927us) server_negotiation.cc:407] Connection header received
0810 01:03:48.542312 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:48.542318 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:48.542426 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:48.542521 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:48.546957 (+ 4436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.548031 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.552172 (+ 4141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.552977 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.553095 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:48.570989 (+ 17894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:48.570997 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:48.571005 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:48.571053 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:48.574902 (+ 3849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.587140 (+ 12238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.587146 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.587149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.587350 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.589898 (+ 2548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.589906 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.589910 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.590709 (+ 799us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:48.591000 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:48.591573 (+ 573us) server_negotiation.cc:299] Negotiation successful
0810 01:03:48.591848 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 01:03:48.600564 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.541896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57912 (local address 127.20.81.189:44735)
0810 01:03:48.542091 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:48.542098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:48.550003 (+ 7905us) server_negotiation.cc:407] Connection header received
0810 01:03:48.550376 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:48.550382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:48.550497 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:48.550571 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:48.573709 (+ 23138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.574909 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.576830 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.577471 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.577611 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:48.590800 (+ 13189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:48.590806 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:48.590810 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:48.590853 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:48.594370 (+ 3517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.597360 (+ 2990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.597367 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.597372 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.597580 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.598285 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.598291 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.598294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.598692 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:48.598921 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:48.599942 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0810 01:03:48.600184 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
Aug 10 01:03:48 dist-test-slave-gp6n krb5kdc[14322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323827, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.81.189@KRBTEST.COM for kudu/127.20.81.188@KRBTEST.COM
I20260810 01:03:48.618930 14589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.550369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41734 (local address 127.20.81.188:37401)
0810 01:03:48.550565 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:48.550570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:48.569315 (+ 18745us) server_negotiation.cc:407] Connection header received
0810 01:03:48.569581 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:48.569589 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:48.569731 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:48.569844 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:48.574975 (+ 5131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.576397 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.578977 (+ 2580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.579703 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.579845 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:48.610661 (+ 30816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:48.610669 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:48.610676 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:48.610717 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:48.614833 (+ 4116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.615825 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.615831 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.615835 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.615990 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.616698 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.616716 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.616719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.617085 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:48.617314 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:48.618280 (+ 966us) server_negotiation.cc:299] Negotiation successful
0810 01:03:48.618553 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":247,"server-negotiator.queue_time_us":75}
I20260810 01:03:48.625136 14501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.592218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.81.188:37401 (local address 127.0.0.1:41740)
0810 01:03:48.592698 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0810 01:03:48.592722 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 01:03:48.592851 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 01:03:48.594431 (+ 1580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 01:03:48.594439 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 01:03:48.595114 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0810 01:03:48.595387 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:48.595403 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.597437 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.597445 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:48.599406 (+ 1961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 01:03:48.599425 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.601420 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.601429 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 01:03:48.601817 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:48.603127 (+ 1310us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 01:03:48.603150 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 01:03:48.606376 (+ 3226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 01:03:48.619881 (+ 13505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:48.619888 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:48.619893 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:48.620475 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:48.621726 (+ 1251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 01:03:48.621731 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 01:03:48.621735 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 01:03:48.621912 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 01:03:48.624041 (+ 2129us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 01:03:48.624046 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 01:03:48.624220 (+ 174us) client_negotiation.cc:769] Sending connection context
0810 01:03:48.624355 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 01:03:48.624677 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316}
I20260810 01:03:48.627074 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.593412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41740 (local address 127.20.81.188:37401)
0810 01:03:48.593664 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:48.593672 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:48.593695 (+ 23us) server_negotiation.cc:407] Connection header received
0810 01:03:48.593761 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:48.593769 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:48.593879 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:48.594002 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:48.595668 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.597197 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.599630 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.600943 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.601102 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:48.606628 (+ 5526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 01:03:48.606635 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 01:03:48.606640 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 01:03:48.606681 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 01:03:48.619512 (+ 12831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.620878 (+ 1366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.620885 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.620888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.621413 (+ 525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 01:03:48.622242 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 01:03:48.622247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 01:03:48.622252 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 01:03:48.623838 (+ 1586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 01:03:48.625734 (+ 1896us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:48.625940 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 01:03:48.626390 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 01:03:48.634099 14497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 677cef0a85514ab7b4cdb768ce6e69bc: Bootstrap starting.
I20260810 01:03:48.635176 14555 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8b4b685fb46e4a40b742d270de0e7cf9 [term 1 FOLLOWER]: Refusing update from remote peer 3d430ff6bcbd404299b1f1d8369ff53f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:03:48.636191 14611 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b4b685fb46e4a40b742d270de0e7cf9" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:03:48.644405 14497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 677cef0a85514ab7b4cdb768ce6e69bc: Neither blocks nor log segments found. Creating new log.
I20260810 01:03:48.645838 14497 log.cc:824] T 00000000000000000000000000000000 P 677cef0a85514ab7b4cdb768ce6e69bc: Log is configured to *not* fsync() on all Append() calls
W20260810 01:03:48.648648 14532 tablet.cc:2369] T 00000000000000000000000000000000 P 8b4b685fb46e4a40b742d270de0e7cf9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:03:48.654942 14636 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 01:03:48.657917 14497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 677cef0a85514ab7b4cdb768ce6e69bc: No bootstrap required, opened a new log
I20260810 01:03:48.658957 14341 master_runner.cc:186] Error getting master registration for 127.20.81.189:44735: OK, instance_id { permanent_uuid: "677cef0a85514ab7b4cdb768ce6e69bc" instance_seqno: 1786323827860608 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 01:03:48.671229 14341 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:03:48.674084 14593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b4b685fb46e4a40b742d270de0e7cf9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d430ff6bcbd404299b1f1d8369ff53f. Latest consensus state: current_term: 1 leader_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43339 } } peers { permanent_uuid: "677cef0a85514ab7b4cdb768ce6e69bc" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } } peers { permanent_uuid: "8b4b685fb46e4a40b742d270de0e7cf9" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } } }
I20260810 01:03:48.674960 14593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b4b685fb46e4a40b742d270de0e7cf9 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:03:48.676878 14634 mvcc.cc:205] Tried to move back new op lower bound from 7316782402056392704 to 7316782401726611456. Current Snapshot: MvccSnapshot[applied={T|T < 7316782402056392704}]
I20260810 01:03:48.681109 14497 raft_consensus.cc:348] T 00000000000000000000000000000000 P 677cef0a85514ab7b4cdb768ce6e69bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43339 } } peers { permanent_uuid: "677cef0a85514ab7b4cdb768ce6e69bc" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } } peers { permanent_uuid: "8b4b685fb46e4a40b742d270de0e7cf9" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } }
I20260810 01:03:48.684701 14497 raft_consensus.cc:729] T 00000000000000000000000000000000 P 677cef0a85514ab7b4cdb768ce6e69bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 677cef0a85514ab7b4cdb768ce6e69bc, State: Initialized, Role: FOLLOWER
I20260810 01:03:48.686262 14497 consensus_queue.cc:260] T 00000000000000000000000000000000 P 677cef0a85514ab7b4cdb768ce6e69bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43339 } } peers { permanent_uuid: "677cef0a85514ab7b4cdb768ce6e69bc" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } } peers { permanent_uuid: "8b4b685fb46e4a40b742d270de0e7cf9" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } }
I20260810 01:03:48.690282 14593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b4b685fb46e4a40b742d270de0e7cf9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43339 } } peers { permanent_uuid: "677cef0a85514ab7b4cdb768ce6e69bc" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } } peers { permanent_uuid: "8b4b685fb46e4a40b742d270de0e7cf9" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } } }
I20260810 01:03:48.690551 14593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b4b685fb46e4a40b742d270de0e7cf9 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:03:48.691161 14623 catalog_manager.cc:1347] Generated new cluster ID: 550fdb82930c496592bc5e6151fe801b
I20260810 01:03:48.691336 14623 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 01:03:48.691159 14611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43339 } } peers { permanent_uuid: "677cef0a85514ab7b4cdb768ce6e69bc" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } } peers { permanent_uuid: "8b4b685fb46e4a40b742d270de0e7cf9" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } } }
I20260810 01:03:48.691476 14611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:48.692310 14640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 677cef0a85514ab7b4cdb768ce6e69bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43339 } } peers { permanent_uuid: "677cef0a85514ab7b4cdb768ce6e69bc" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } } peers { permanent_uuid: "8b4b685fb46e4a40b742d270de0e7cf9" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } } }
I20260810 01:03:48.693959 14640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 677cef0a85514ab7b4cdb768ce6e69bc [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:03:48.693173 14467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "677cef0a85514ab7b4cdb768ce6e69bc"
I20260810 01:03:48.693820 14466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 677cef0a85514ab7b4cdb768ce6e69bc [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:03:48.697850 14466 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 677cef0a85514ab7b4cdb768ce6e69bc [term 1 FOLLOWER]: Refusing update from remote peer 3d430ff6bcbd404299b1f1d8369ff53f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:03:48.698755 14611 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f [LEADER]: Connected to new peer: Peer: permanent_uuid: "677cef0a85514ab7b4cdb768ce6e69bc" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:03:48.701893 14467 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 677cef0a85514ab7b4cdb768ce6e69bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d430ff6bcbd404299b1f1d8369ff53f in term 1.
I20260810 01:03:48.693131 14497 sys_catalog.cc:564] T 00000000000000000000000000000000 P 677cef0a85514ab7b4cdb768ce6e69bc [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:03:48.719367 14623 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 01:03:48.721647 14623 catalog_manager.cc:1504] Loading token signing keys...
I20260810 01:03:48.755602 14640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 677cef0a85514ab7b4cdb768ce6e69bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d430ff6bcbd404299b1f1d8369ff53f. Latest consensus state: current_term: 1 leader_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43339 } } peers { permanent_uuid: "677cef0a85514ab7b4cdb768ce6e69bc" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } } peers { permanent_uuid: "8b4b685fb46e4a40b742d270de0e7cf9" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } } }
I20260810 01:03:48.755610 14641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 677cef0a85514ab7b4cdb768ce6e69bc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43339 } } peers { permanent_uuid: "677cef0a85514ab7b4cdb768ce6e69bc" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } } peers { permanent_uuid: "8b4b685fb46e4a40b742d270de0e7cf9" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } } }
I20260810 01:03:48.756031 14640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 677cef0a85514ab7b4cdb768ce6e69bc [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:03:48.756031 14641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 677cef0a85514ab7b4cdb768ce6e69bc [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:03:48.759935 14611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d430ff6bcbd404299b1f1d8369ff53f" member_type: VOTER last_known_addr { host: "127.20.81.190" port: 43339 } } peers { permanent_uuid: "677cef0a85514ab7b4cdb768ce6e69bc" member_type: VOTER last_known_addr { host: "127.20.81.189" port: 44735 } } peers { permanent_uuid: "8b4b685fb46e4a40b742d270de0e7cf9" member_type: VOTER last_known_addr { host: "127.20.81.188" port: 37401 } } }
I20260810 01:03:48.760195 14611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f [sys.catalog]: This master's current role is: LEADER
I20260810 01:03:48.765470 14623 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d430ff6bcbd404299b1f1d8369ff53f: Generated new TSK 0
I20260810 01:03:48.766569 14623 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 01:03:48.797763 14658 catalog_manager.cc:1259] Loaded cluster ID: 550fdb82930c496592bc5e6151fe801b
I20260810 01:03:48.797888 14658 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 677cef0a85514ab7b4cdb768ce6e69bc: loading cluster ID for follower catalog manager: success
I20260810 01:03:48.801690 14658 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 677cef0a85514ab7b4cdb768ce6e69bc: acquiring CA information for follower catalog manager: success
I20260810 01:03:48.804258 14658 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 677cef0a85514ab7b4cdb768ce6e69bc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 01:03:48.935983 14588 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 01:03:48.933279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53218 (local address 127.20.81.190:43339)
0810 01:03:48.933490 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:48.933497 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:48.933924 (+ 427us) server_negotiation.cc:407] Connection header received
0810 01:03:48.934229 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:48.934236 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:48.934330 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:48.934436 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:48.935679 (+ 1243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53218: BlockingRecv error: recv got EOF from 127.0.0.1:53218 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260810 01:03:48.944008 14503 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 01:03:48.940793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57920 (local address 127.20.81.189:44735)
0810 01:03:48.940991 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:48.940997 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:48.942314 (+ 1317us) server_negotiation.cc:407] Connection header received
0810 01:03:48.942595 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:48.942601 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:48.942703 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:48.942786 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:48.943759 (+ 973us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57920: BlockingRecv error: recv got EOF from 127.0.0.1:57920 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260810 01:03:48.946466 14585 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 01:03:48.941731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41748 (local address 127.20.81.188:37401)
0810 01:03:48.941905 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:48.941911 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:48.944755 (+ 2844us) server_negotiation.cc:407] Connection header received
0810 01:03:48.945030 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:48.945038 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:48.945130 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:48.945204 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:48.946213 (+ 1009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41748: BlockingRecv error: recv got EOF from 127.0.0.1:41748 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 01:03:48.996016 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:48.980322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57928 (local address 127.20.81.189:44735)
0810 01:03:48.980523 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:48.980529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:48.987486 (+ 6957us) server_negotiation.cc:407] Connection header received
0810 01:03:48.988060 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:48.988067 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:48.988271 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:48.988428 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:48.989153 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.990327 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.992760 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:48.993450 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:48.993548 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:48.994322 (+ 774us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:48.995077 (+ 755us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:48.995204 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:48.995540 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 01:03:48.995638 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260810 01:03:49.039433 14585 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 01:03:49.029964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41758 (local address 127.20.81.188:37401)
0810 01:03:49.030165 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:49.030172 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:49.037134 (+ 6962us) server_negotiation.cc:407] Connection header received
0810 01:03:49.037693 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:49.037700 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:49.037833 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:49.037929 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:49.039118 (+ 1189us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41758: BlockingRecv error: recv got EOF from 127.0.0.1:41758 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 01:03:49.088773 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:49.073900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57932 (local address 127.20.81.189:44735)
0810 01:03:49.074094 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:49.074101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:49.081075 (+ 6974us) server_negotiation.cc:407] Connection header received
0810 01:03:49.081636 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:49.081643 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:49.081766 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:49.081841 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:49.082707 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.083965 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.086253 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.086993 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.087110 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:49.087830 (+ 720us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:49.088028 (+ 198us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:49.088096 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:49.088341 (+ 245us) server_negotiation.cc:299] Negotiation successful
0810 01:03:49.088439 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260810 01:03:49.144774 14585 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 01:03:49.135103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41768 (local address 127.20.81.188:37401)
0810 01:03:49.135370 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:49.135376 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:49.142512 (+ 7136us) server_negotiation.cc:407] Connection header received
0810 01:03:49.143111 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:49.143118 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:49.143234 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:49.143327 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:49.144508 (+ 1181us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41768: BlockingRecv error: recv got EOF from 127.0.0.1:41768 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 01:03:49.199605 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:49.184783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57944 (local address 127.20.81.189:44735)
0810 01:03:49.185002 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:49.185009 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:49.191983 (+ 6974us) server_negotiation.cc:407] Connection header received
0810 01:03:49.192561 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:49.192569 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:49.192685 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:49.192762 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:49.193544 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.194819 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.197128 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.197771 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.197889 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:49.198585 (+ 696us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:49.198770 (+ 185us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:49.198843 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:49.199112 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 01:03:49.199205 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260810 01:03:49.229652 14585 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 01:03:49.227200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41772 (local address 127.20.81.188:37401)
0810 01:03:49.227404 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:49.227412 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:49.228004 (+ 592us) server_negotiation.cc:407] Connection header received
0810 01:03:49.228202 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:49.228207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:49.228321 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:49.228435 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:49.229400 (+ 965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41772: BlockingRecv error: recv got EOF from 127.0.0.1:41772 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 01:03:49.244475 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:49.237106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57960 (local address 127.20.81.189:44735)
0810 01:03:49.237363 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:49.237370 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:49.237960 (+ 590us) server_negotiation.cc:407] Connection header received
0810 01:03:49.238254 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:49.238261 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:49.238370 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:49.238503 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:49.239060 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.240282 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.242237 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.242996 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.243137 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:49.243649 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:49.243786 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:49.243899 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:49.244096 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 01:03:49.244179 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
W20260810 01:03:49.260691 14585 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 01:03:49.252393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41782 (local address 127.20.81.188:37401)
0810 01:03:49.252695 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:49.252701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:49.258867 (+ 6166us) server_negotiation.cc:407] Connection header received
0810 01:03:49.259228 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:49.259234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:49.259358 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:49.259484 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:49.260436 (+ 952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41782: BlockingRecv error: recv got EOF from 127.0.0.1:41782 (error 108)
Metrics: {"server-negotiator.queue_time_us":157}
I20260810 01:03:49.279832 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:49.272302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57976 (local address 127.20.81.189:44735)
0810 01:03:49.272515 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:49.272521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:49.273487 (+ 966us) server_negotiation.cc:407] Connection header received
0810 01:03:49.273696 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:49.273703 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:49.273799 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:49.273913 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:49.274433 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.275545 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.277583 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.278239 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.278379 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:49.278941 (+ 562us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:49.279103 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:49.279206 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:49.279387 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 01:03:49.279469 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260810 01:03:49.294313 14585 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 01:03:49.292055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41794 (local address 127.20.81.188:37401)
0810 01:03:49.292230 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:49.292236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:49.292780 (+ 544us) server_negotiation.cc:407] Connection header received
0810 01:03:49.292934 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:49.292940 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:49.293047 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:49.293160 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:49.294040 (+ 880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41794: BlockingRecv error: recv got EOF from 127.0.0.1:41794 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 01:03:49.321388 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:49.313691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57990 (local address 127.20.81.189:44735)
0810 01:03:49.313875 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:49.313881 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:49.314826 (+ 945us) server_negotiation.cc:407] Connection header received
0810 01:03:49.315003 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:49.315012 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:49.315133 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:49.315255 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:49.315793 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.316995 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.319054 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.319777 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.319941 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:49.320545 (+ 604us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:49.320689 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:49.320798 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:49.320983 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 01:03:49.321087 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260810 01:03:49.337203 14585 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 01:03:49.328540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41808 (local address 127.20.81.188:37401)
0810 01:03:49.328741 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:49.328748 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:49.335583 (+ 6835us) server_negotiation.cc:407] Connection header received
0810 01:03:49.335773 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:49.335780 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:49.335896 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:49.336029 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:49.336977 (+ 948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41808: BlockingRecv error: recv got EOF from 127.0.0.1:41808 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 01:03:49.379722 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:49.373616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58004 (local address 127.20.81.189:44735)
0810 01:03:49.373880 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:49.373887 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:49.374329 (+ 442us) server_negotiation.cc:407] Connection header received
0810 01:03:49.374579 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:49.374585 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:49.374716 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:49.374829 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:49.375237 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.376331 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.377638 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.378201 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.378332 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:49.378907 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:49.379051 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:49.379141 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:49.379294 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 01:03:49.379376 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260810 01:03:49.387977 14585 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 01:03:49.385915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41824 (local address 127.20.81.188:37401)
0810 01:03:49.386156 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:49.386163 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:49.386398 (+ 235us) server_negotiation.cc:407] Connection header received
0810 01:03:49.386751 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:49.386759 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:49.386850 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:49.386936 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:49.387732 (+ 796us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41824: BlockingRecv error: recv got EOF from 127.0.0.1:41824 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
I20260810 01:03:49.460758 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:49.455723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58016 (local address 127.20.81.189:44735)
0810 01:03:49.455970 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:49.455976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:49.456299 (+ 323us) server_negotiation.cc:407] Connection header received
0810 01:03:49.456514 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:49.456518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:49.456603 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:49.456686 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:49.457055 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.457782 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.459180 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.459622 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.459713 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:49.460068 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:49.460174 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:49.460240 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:49.460378 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 01:03:49.460451 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260810 01:03:49.468335 14585 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 01:03:49.466618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41826 (local address 127.20.81.188:37401)
0810 01:03:49.466867 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:49.466874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:49.467096 (+ 222us) server_negotiation.cc:407] Connection header received
0810 01:03:49.467231 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:49.467235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:49.467309 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:49.467396 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 01:03:49.468129 (+ 733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41826: BlockingRecv error: recv got EOF from 127.0.0.1:41826 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
I20260810 01:03:49.570362 14608 catalog_manager.cc:1259] Loaded cluster ID: 550fdb82930c496592bc5e6151fe801b
I20260810 01:03:49.570503 14608 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8b4b685fb46e4a40b742d270de0e7cf9: loading cluster ID for follower catalog manager: success
I20260810 01:03:49.573031 14608 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8b4b685fb46e4a40b742d270de0e7cf9: acquiring CA information for follower catalog manager: success
I20260810 01:03:49.575009 14608 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8b4b685fb46e4a40b742d270de0e7cf9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 01:03:49.608381 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:49.603058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58018 (local address 127.20.81.189:44735)
0810 01:03:49.603303 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:49.603311 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:49.603679 (+ 368us) server_negotiation.cc:407] Connection header received
0810 01:03:49.603908 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:49.603918 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:49.603988 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:49.604090 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:49.604516 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.605229 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.606610 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.607069 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.607216 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:49.607655 (+ 439us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:49.607770 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:49.607870 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:49.608024 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 01:03:49.608107 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 01:03:49.620245 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:49.614530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41836 (local address 127.20.81.188:37401)
0810 01:03:49.614790 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:49.614797 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:49.615085 (+ 288us) server_negotiation.cc:407] Connection header received
0810 01:03:49.615180 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:49.615184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:49.615343 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:49.615448 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:49.615880 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.616608 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.617937 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.618352 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.618586 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:49.619073 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:49.619621 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:49.619811 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:49.619868 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 01:03:49.619953 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260810 01:03:49.633231 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:49.626695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58034 (local address 127.20.81.189:44735)
0810 01:03:49.626901 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:49.626909 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:49.627113 (+ 204us) server_negotiation.cc:407] Connection header received
0810 01:03:49.627455 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:49.627468 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:49.627560 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:49.627626 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:49.628422 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.629127 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.630819 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.631507 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.631632 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:49.632468 (+ 836us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:49.632592 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:49.632685 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:49.632898 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 01:03:49.632977 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 01:03:49.634225 14776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:49.625815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53232 (local address 127.20.81.190:43339)
0810 01:03:49.626377 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:49.626383 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:49.626571 (+ 188us) server_negotiation.cc:407] Connection header received
0810 01:03:49.626964 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:49.626970 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:49.627197 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:49.627378 (+ 181us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:49.628116 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.629319 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.631353 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.631971 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.632090 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:49.632833 (+ 743us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:49.633268 (+ 435us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:49.633379 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:49.633819 (+ 440us) server_negotiation.cc:299] Negotiation successful
0810 01:03:49.633903 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":290,"threads_started":1}
I20260810 01:03:49.635213 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 01:03:49.627264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41842 (local address 127.20.81.188:37401)
0810 01:03:49.627489 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 01:03:49.627497 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 01:03:49.628096 (+ 599us) server_negotiation.cc:407] Connection header received
0810 01:03:49.628393 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 01:03:49.628398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 01:03:49.628494 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 01:03:49.628557 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 01:03:49.629228 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.630298 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.633059 (+ 2761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 01:03:49.633729 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:03:49.633825 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 01:03:49.634483 (+ 658us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 01:03:49.634616 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 01:03:49.634666 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 01:03:49.634864 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 01:03:49.634940 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 01:03:49.639134 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 14341
I20260810 01:03:49.640137 14405 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:03:49.751063 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 14341
W20260810 01:03:49.778865 14530 connection.cc:535] server connection from 127.0.0.1:41732 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 01:03:49.779543 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 14419
I20260810 01:03:49.780819 14491 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:03:49.892309 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 14419
I20260810 01:03:49.916139 20806 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5NFMaT/build/asan/bin/kudu pid 14507
I20260810 01:03:49.917411 14577 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:03:50.027881 20806 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5NFMaT/build/asan/bin/kudu with pid 14507
2026-08-10T01:03:50Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4740 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9499 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (384295 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST